Posts Tagged ‘Computer Troubles’

November 14, 2015

Data Loss

Someone once told me the problem with health insurance is you don’t know how good yours is until you need it, and by then it’s too late. The same can be said about computer backup recovery systems.

I’ve been using crash plan for two years now. Up until now I’ve been quite happy with our setup. My data is backed up in triplicate. One external hard drive drive (E:) contains all the raw images, directly off the camera, in the same directory structure the camera creates. Another drive (F:) has all my images organized, so I can easily find the photos I’m looking for. I’m using Crashplan to back up both those drives to another local hard drive (G:) and also to the cloud. More on the cloud later. I figured I had to be safe from data loss, right? How could three copies of my data not be sufficient?

I should mention that before creating the local backup instance on drive G:, I had tested the crash plan’s cloud backup. I say that as though the test was intentional, and not because I accidentally deleted a directory. Regardless, recovering the deleted files was easy peasy lemon squeezy so it never occurred to me to also test the local backup instance. That mistake is on me.

On October 17th my E: drive failed. All the light weight solutions – chkdsk, restarting, etc – where to no avail. My computer happily told me the disk was unreadable and suggested reformatting. I decided to not waste too much time trying to repair the drive. After all this was exactly the use case for the local crash plan backup. I reformated the drive and began the process of restoring over 296,224 files.

I got back 296,224 “Unknown problem” error responses.

At this point I wasn’t expecting to experience much, if any, data loss. I could still pay the $300 and recover from the cloud, and I had my F: drive which should be the same files. I say “should” because the two drives do get out of sink some times. I previously wrote a java program to run through the directories to warn me when this happens, but I couldn’t remember the last time I ran the job.

I couldn’t figure out what went wrong from the logs so I contacted tech support. Tech support theorized it was a known bug that affect NTFS file systems on Windows computer when the drive was reformatted. This was extremely frustrating to hear as NTFS is the default file system on windows, and recovering from a dead drive was, again, one of the primary uses cases for crashplan! Tech support’s suggestions included (1) try a different operating system, (2) reformat the drive to a different file system, and (3) downloand the files piecemeal. Of those, (2) was the least ridiculous. I reformatted the drive to ExFat and tried again.

I got back 411 files. A handful of iPhone photos and a bunch more “Unknown problem” error messages.

But this time I could tell there was still an error with the drive. The 2 TB drive was showing as full with just a handful of data. I reformatted again. Reformat failed. Repeat, repeat, repeat. After the fourth reformatting failed I purchased a new hard drive.

I got back 34k files.

The troubling thing now was crash plan was failing silently. There was no error given. The only indication that something had gone wrong was the fact that I had only gotten back a tenth of my files. I tried again, more silent failures. At this point my confidence that I was going to get back all my data was waning so I started looking into the cloud.

I had two options when it came to restoring from the cloud. The first was to download all 2TBs worth of data in 500 MB chunks – 4000 chunks to be exact! The second was to pay $300 to “Restore to Door”. I had originally thought “Restore to door” meant they send you a hard drive with all your files in tact. Nope, they send you a local crash plan instance you can restore from. Basically I would have another copy of my G: drive. That didn’t leave me much hope that it would fair any better.

I restored again, this time a smaller subset of photos from my local backup. Success! Restored again, more success! I was able to partition my data into five chunks and restore each chunk without issue. The process took two weeks, as I kept getting stuck waiting for crashplan maintenance modes; deep pruning, synchronizing, etc. The deep prune itself took four days to complete.

In the end I lost just 16 photos. I’m not happy about that, but I can live with it.

My lessons learned:

I’m not the typical use case Crash Plan was designed for. Crashplan is sort of a light weight version control system in addition to backup engine. It keeps multiple versions of your files (as many as you specify), and are constantly scanning your file system looking for recent changes. In doing so they make the design decision to focus on recently changed files. That makes a lot of sense if your backing up your working directory. You probably want/need the latest version of any paper your writing, or program your coding. It makes less sense if you’re backing up an archive full of photos. I want at least one version of every photo backed up. I can always re-edit them, but I can’t re-shoot them!

I have more data than the typical Crash Plan user. When I was searching through the forums looking for tips on how to speed the process up, (4 days to deep prune, are you kidding me?!), I found a number of folks with similar problems, each with large data sets. Between all our home computers we had backed up 3TBs worth of data. I had already busted the memory of the app as high as their sample recommendations go, and went even higher during this process. I’m starting to reach the point where crashplan just cannot hold everything in memory it needs to. When that happens with any program performance drops off a cliff. When crash plan runs there’s not a lot else I can do with the computer.

The Verdict on Crashplan:

Obviously the continuing to fail drive was not Crashplan’s fault, and I was able to recover almost all of my files. Still, taking two weeks to recover a hard drive seems a bit excessive. I take a lot photos, and I don’t expect that to change any time soon. Crashplan just may not be right for my use case. I have a little over a year left on my crashplan subscription and I see no reason to jump ship now. I may look into alternative back up options when the end of the subscription nears.

October 23, 2013

De·Fend·Ed

Exhausted, Relieved, Elated, Happy, Drained, Ecstatic. Defended. De-Fend-Ed.

At 4:11 PM my dissertation committee wrapped up their discussion. I had passed with “flying colors”. It was surreal hearing those words. I expected to pass as all grad students who finish their theses are conditioned to expect. What I wasn’t sure on was how my thesis stacked up against the expectation, or how much work my committee would request I do before signing off on it. I was not expecting to walk out, title page in hand, with two signatures already and a third (remote) member ready to sign once it arrived to him. I was hoping, but not daring to expect.

My plan for the day was to go to bed early – no later than 11 pm. I’d drop off Nicki at daycare, then Domingo and I would drive down to Santa Cruz two to three hours before my actual defense at 2 pm. I’d have time to pay the filing fee when I arrived, do a full run through of my presentation before my defense.

The actual day turned out very different.

I’m a believer that impressions matter, and wanted to wanted to project a professional image so I decided to dress “business casual.” I had read everything from full suits required, to not to worry about appearances. Computer science tends to be a very causal field as far as attire goes, so I opted for the “dress one step above your interviewers” (or in this case committee members) rule. When we were packing last week for our move, I made sure I had an outfit I liked, that I could feel confident in. The trouble is, I didn’t set it aside. I expected to be unpacked by now (ha!). That outfit? Buried in one of my suitcases. So at around 11:30 I was madly searching for anything that fit the bill and wasn’t too wrinkled. Luckily I found a pale pink shirt and dark gray sweater that were relatively wrinkle free.

The last step before bed was to print my thesis. I had read that the committee may have specific questions about specific figures, and having a hard copy makes things easier. (My committee did have such questions.)

Last time I tried to print my dissertation it took 54 minutes, 25.9 seconds. Yes, I timed it. I’m a PhD, we do weird things like that. That was back when it was 172 pages, these days it’s 200 pages. That’s a rate of about one page per 18 seconds. At the time I thought the problem was my cheapie printer running out of buffering space. It turns out it’s my laptop and my laptop is dying. It’s having memory corruption errors. I know this because it blue screened of death on me TWICE while trying to print, and hung once and required a full disk check on boot up. After trying for over an hour I abandoned the idea of having a printed copy. But I couldn’t just go to bed at that point. I had no other presentation device. I needed to know whether my laptop would boot or I would be making an emergency run to BestBuy in the morning. That meant I had to wait through the full check disc process. It turned out that as long as I didn’t attempt to print, or open up a web browser, my computer had enough memory to give my presentation. I went to bed a little after 1.

Then Nick decided it was time to get up at 4. And again at 5:30. And finally at 6:30.

Daycare drop off went a little longer than expected. Nicki is handling the switch like a boss. The recommendation was four visit days. The second day she was content to play by herself away from me for twenty minute intervals. The third day she was good for a few hours. Her teachers have been so impressed by how quickly she’s been adapting. No Tears! Still I wanted to wait until snack time to leave, since she’s usually sufficiently distracted by tasty goodness that she doesn’t notice my exit. Snack time was a little delayed, and apparently I have been a little too consistent in my exit. Nicki noticed the pattern. She started to walk off in the direction of snack, but then changed her mind and came flying back into my arms. Little kids are excellent at deducing patterns when they want to be!

Back at home Domingo had not only managed to print my thesis in under five minutes, but had swung by target and picked up brownies and cookies for my committee. Have I mentioned before how absolutely amazing and what a life saver he is?

I finally found the phone number to order coffee for my committee members, a quick shower, and it was time to leave. Despite everything we were still making good time and would be on campus two hours early.

About 15 minutes into the journey to campus traffic stopped. There is exactly one route through the mountains to UC Santa Cruz – highway 17. There was a brush fire and the right lane was closed. Traffic was backed up for miles. We arrived at campus with less than an hour to go. A quick bite of lunch, a few minutes to set-up and it was go time.

October 4, 2011

I Love my Systems Guy

I mentioned a few days ago about being stressed at school. I talked with my adviser about the setback I encountered, and we came up with a game plan to get around it. All was good. Ha! How naïve of me to think so. Since then, I had a massive hard drive issues that threatened to derail me further. Actually, I had two hard drive issues.

The first was with my external hard drive that has all all my research data on for the project I’ve been working. I know, I know, I should have a backup – but that’s hard when you’re talking about Terabytes of data. After I restarted the computer the disk would not mount. I could not access any of the data. Campus technical support couldn’t figure it out. There suggestion was to reformat the drive, but that would delete all my data. I got so stressed out about it, I ended up with a horrible migraine Saturday night. That’s when my wonderful husband saved the day.

Domingo realized the computer couldn’t read the drive because of an incompatibility with the BIOS. The BIOS controls what the computer does when it’s turned on. It meant two things. Reformatting would only work until the computer was turned off. Once the computer was booted again, the incompatibility with the BIOS would surface again and the drive would no longer be accessible. And secondly, there was a possibility there was no problem with the hard drive, and that my data was still in tact. Sunday, we took the two hour trip to campus, and he confirmed the problem. We brought the drive back home, and since then have been transferring the data to an external drive that my computer at school will be compatible with. Had Domingo not figured out the problem I not only would have lost my data, but I would keep loosing the drive every time I restarted my computer!

Then, yesterday, I had a personal hard drive crash. I booted my desktop to get a disk head crash error message. Disk head crashes are often unrecoverable. Sometimes the drive will still work, but it’s usually prone to more crashes. In this case, I was able to start the computer just long enough to copy all the data off before it crashed for good.

I am so glad to have survived the week without any data loss.