Someone once told me the problem with health insurance is you don’t know how good yours is until you need it, and by then it’s too late. The same can be said about computer backup recovery systems.
I’ve been using crash plan for two years now. Up until now I’ve been quite happy with our setup. My data is backed up in triplicate. One external hard drive drive (E:) contains all the raw images, directly off the camera, in the same directory structure the camera creates. Another drive (F:) has all my images organized, so I can easily find the photos I’m looking for. I’m using Crashplan to back up both those drives to another local hard drive (G:) and also to the cloud. More on the cloud later. I figured I had to be safe from data loss, right? How could three copies of my data not be sufficient?
I should mention that before creating the local backup instance on drive G:, I had tested the crash plan’s cloud backup. I say that as though the test was intentional, and not because I accidentally deleted a directory. Regardless, recovering the deleted files was easy peasy lemon squeezy so it never occurred to me to also test the local backup instance. That mistake is on me.
On October 17th my E: drive failed. All the light weight solutions – chkdsk, restarting, etc – where to no avail. My computer happily told me the disk was unreadable and suggested reformatting. I decided to not waste too much time trying to repair the drive. After all this was exactly the use case for the local crash plan backup. I reformated the drive and began the process of restoring over 296,224 files.
I got back 296,224 “Unknown problem” error responses.
At this point I wasn’t expecting to experience much, if any, data loss. I could still pay the $300 and recover from the cloud, and I had my F: drive which should be the same files. I say “should” because the two drives do get out of sink some times. I previously wrote a java program to run through the directories to warn me when this happens, but I couldn’t remember the last time I ran the job.
I couldn’t figure out what went wrong from the logs so I contacted tech support. Tech support theorized it was a known bug that affect NTFS file systems on Windows computer when the drive was reformatted. This was extremely frustrating to hear as NTFS is the default file system on windows, and recovering from a dead drive was, again, one of the primary uses cases for crashplan! Tech support’s suggestions included (1) try a different operating system, (2) reformat the drive to a different file system, and (3) downloand the files piecemeal. Of those, (2) was the least ridiculous. I reformatted the drive to ExFat and tried again.
I got back 411 files. A handful of iPhone photos and a bunch more “Unknown problem” error messages.
But this time I could tell there was still an error with the drive. The 2 TB drive was showing as full with just a handful of data. I reformatted again. Reformat failed. Repeat, repeat, repeat. After the fourth reformatting failed I purchased a new hard drive.
I got back 34k files.
The troubling thing now was crash plan was failing silently. There was no error given. The only indication that something had gone wrong was the fact that I had only gotten back a tenth of my files. I tried again, more silent failures. At this point my confidence that I was going to get back all my data was waning so I started looking into the cloud.
I had two options when it came to restoring from the cloud. The first was to download all 2TBs worth of data in 500 MB chunks – 4000 chunks to be exact! The second was to pay $300 to “Restore to Door”. I had originally thought “Restore to door” meant they send you a hard drive with all your files in tact. Nope, they send you a local crash plan instance you can restore from. Basically I would have another copy of my G: drive. That didn’t leave me much hope that it would fair any better.
I restored again, this time a smaller subset of photos from my local backup. Success! Restored again, more success! I was able to partition my data into five chunks and restore each chunk without issue. The process took two weeks, as I kept getting stuck waiting for crashplan maintenance modes; deep pruning, synchronizing, etc. The deep prune itself took four days to complete.
In the end I lost just 16 photos. I’m not happy about that, but I can live with it.
My lessons learned:
I’m not the typical use case Crash Plan was designed for. Crashplan is sort of a light weight version control system in addition to backup engine. It keeps multiple versions of your files (as many as you specify), and are constantly scanning your file system looking for recent changes. In doing so they make the design decision to focus on recently changed files. That makes a lot of sense if your backing up your working directory. You probably want/need the latest version of any paper your writing, or program your coding. It makes less sense if you’re backing up an archive full of photos. I want at least one version of every photo backed up. I can always re-edit them, but I can’t re-shoot them!
I have more data than the typical Crash Plan user. When I was searching through the forums looking for tips on how to speed the process up, (4 days to deep prune, are you kidding me?!), I found a number of folks with similar problems, each with large data sets. Between all our home computers we had backed up 3TBs worth of data. I had already busted the memory of the app as high as their sample recommendations go, and went even higher during this process. I’m starting to reach the point where crashplan just cannot hold everything in memory it needs to. When that happens with any program performance drops off a cliff. When crash plan runs there’s not a lot else I can do with the computer.
The Verdict on Crashplan:
Obviously the continuing to fail drive was not Crashplan’s fault, and I was able to recover almost all of my files. Still, taking two weeks to recover a hard drive seems a bit excessive. I take a lot photos, and I don’t expect that to change any time soon. Crashplan just may not be right for my use case. I have a little over a year left on my crashplan subscription and I see no reason to jump ship now. I may look into alternative back up options when the end of the subscription nears.
Today I did something I rarely like to do: I looked at Black Friday ads before solidifying my wish list. Usually I like to do those tasks in the opposite order to guard against being enticed my deals on items we don’t need or really want. But with November rapidly racing buy, I figured I best get started. As always, I’m hoping to spend as much of my shopping time from the warmth of the couch.
Here’s my list:
New PJs. It may sound silly, but hey, if I can get them on sale, why not? My current set of “PJs” consists of a over-sized free t-shirt and a pair of maternity sweat pants. The only reason I’m wearing the maternity pants is because my last pair of the non maternity pair got a little too thread bare to be considered decent, even for home use. I’m ready to retire my free t-shirt collection in general, starting with a pair of actual PJs. This pair from Gilligan O’Mailey is super soft. Domingo likes character sleep pants, but his last pair did not survive the wash.
Target’s black friday sale includes 40% off all apparel. Target’s been having a number of sales lately, and typically has a fairly large one the three days before Thanksgiving that usually includes apparel. I plan to keep my eyes opened. 40% off is the price to beat.
Ornaments. We’re going from one to three trees this year, two more than past years, which means we’ll need a ton more ornaments. I’m torn between my desire to save, and my desire to set up the trees Thursday night per tradition. I’ll probably try and split the difference: get enough ornaments ahead of time that it makes sense to put the trees up, and get the rest during sales.
On Saturday Target is having a “spend $100, save $50” deal on Christmas decor. So far Target’s been the best place I’ve found nice, durable, inexpensive ornaments for the kid’s trees. These tin balls are pretty great. Like most of the cheaper ornaments on Target, they’re not available online, either to ship or for store pick up. I prefer to do my shopping from the couch, if possible. I am not one to stand in line for the doors to open, and I’d worry the selection will be pretty picked through by the time I make it to the store.
Board Games. For as long as I can remember, $5 board games have been a staple black Friday door buster. I’m hoping to stock up on the classics: Sorry!, Trouble, Chutes and Ladders. They’re suspiciously absent from Target’s flyer, but they’re on Walmart’s. What’s not clear is whether I need to go to the store.
New Phone for Domingo. Domingo and I have gotten to the point where our contract is up on alternating Black Fridays. This is Domingo’s year for a new phone. In the past we waited until the phone died – a mistake that cost us $200!
We’ve seen at least one flyer listing the phone he wants for just a penny for Black Friday, a good $300 cheaper than we could get right now. I’d say it’s not worth the weight! This one I’m confident we could get online from Amazon.
New computer. Both my computers are showing their age. My primary work machine can no longer hold a charge, and constantly loses the wifi connection – both not so useful traits when you’re starting your own company! Two black friday’s (technically Cyber Monday) ago I went for a special buy laptop that was more expensive hoping it would last longer. It did not. This year I plan on buying the cheapest computer which meets the specs I need.
I haven’t done any research on this front. Truth be told I’m dreading the process.
Dollhouse. Nicole’s present from Santa this year is going to be a doll house. She fell in love with my sister’s old Playskool Dollhouse when we visited last labor day, so we’re in the market for something similar.
I liked Babies’ R Us’ You & Me Happy Together, as it reminded me a lot of the PlaySkool house, but the reviews were pretty terrible.
Cricut. Cutting machines are another staple of Black Friday that I always assumed I would get one day. When we moved I had all sorts of ideas for vinyl wall decals. Now that I’m finally ready to pull the trigger on a cutting machine, they are absent from those black friday ads! Argh. File this one in the maybe, but not likely category.
How did I get so far behind? Wasn’t it just October? And now it’s already almost mid November? Adding insult to injury, most of the stores I frequent have already started their holiday deals and I haven’t even figured out when I’m on the hunt for yet!
One thing I know I need more of is ornaments. It’s no secret that I’m quite addicted to Christmas Tree, and Christmas ornaments. Every year I buy at least a couple new ornaments for our tree. Pretty much every year since Domingo and I first started dating he has been teasing me about the ornaments that will one day grace my tree. We’d be at Hallmark store and he’d point to the Twilight Ornament, or the Hanna Montana Ornament. One day, he’d warn, our future kids would want those. The notion of kid centric ornaments on our tree didn’t bother me, per se. I do love me some Dr. Suess, as evident from Nicole’s first birthday party. But I do cringe a little at the idea of tween pop culture on the tree. I could do without vampire romance, thank you very much.
The big driver for ornaments this year is the second tree I’ve been pinning over. As we were discussing where the trees will go in our new home it finally dawned on us; why stop there? We can get a third kid-height, narrow tree for the playroom. The kids can have total say of what goes on the tree, and decorate it to their hearts content. Domingo and I plan to put up the other two trees and decorate them with the fragile ornaments Thanksgiving night after the girls go to sleep, per tradition. Then, the following morning, we’ll let the girls direct the show when setting up their own tree. It’s all the magic of “Santa” bringing the tree, with all the family fun of setting it up together.
Three trees it is!
Yesterday evening I found the multi-colored light tree on Target.com, equivalent to one we really liked when we saw it in person. The website were stacking sales, including “a buy three, get an additional 15% off” ending that night. I did the frugal thing: searched by sale items, sorted by price, and added two $1 ornament hooks to my card for an additional $9 in savings. This morning I couldn’t help myself. I logged back on to target.com and found the tree had a whole new set of stackable sales, and was now about $5 lower. My order, made just hours before, couldn’t be canceled. My two choices were to suck it up, or call the customer help line and ask for a price reduction. I’m sure you can guess which option I picked. The kind customer sales agent refunded my order the requested amount seemingly without verifying the exact price difference.
There could be several reasons why Target issued me a flat price adjustment. Perhaps they have a minimum price for which they want to sell the item, and my price adjustment was still north of it. (If my math is correct, I came out 11 cents ahead of today’s sale price.) Or perhaps they view it as a means of customer retention. In retail there’s a notion of customer lifetime value. It’s often cheaper to retain a customer than to acquire a new one. There’s also a growing field in Customer Analytics, analytics derived from customer data. Target knows a surprising amount about me. They know my credit card information and address from the online order. From there they can easily figure out how many other purchases I’ve made using the same card or shipping to the same address. From my purchase history they can predict things about me, and my future consumer behaviors. They can predict how profitable I’m likely to continue to be. That assessment cam lead to special perks. It’s a practice that’s been around for ages (think Casino “comps”), but is becoming more frequent with the advent of big data.
It’s hard to know for sure what a company considers my worth to be. Eleven cents extra doesn’t tell you much. I tend to think of myself as not a very profitable customer. I like to think I’m pretty good about ferreting out the best possible deals. I’m clearly the kind of person who double checks sale prices after a purchase, and requests price matching even when it’s just a few dollars. Each dollar I save is a dollar the retailer doesn’t earn. The margins on me can’t be very good. Domingo keeps reminding me it’s not just the margins retailers care about, but the reduction of inventory. In that regard I’m a golden goose, what with my three trees and two Halloween costumes per child. In my own analysis, I did point out that I paid slightly more ($5-7 per child) than average.
This year I’m sure the models will tell the retailers I’m anything but frugal. I’m going to try and hold off on the bulk of my holiday decorations until the after sales, but we will need at least some ornaments for our new trees.
After the success of my Labor Predictor, I started thinking about a Miscarriage calculator as a natural follow on idea. I like math, so diving into the statistics generally make me feel better about things that are beyond my control. I’ve also been around long enough to know that’s not necessarily true for everyone, or even most people. In my passive observations, there didn’t seem to be enough interest in such an app to make creating it worth while. I tabled the idea.
Then a dear friend suffered a loss.
As devastating as the loss itself can be, uncertainty can make it so much worse. On bad days my friend worried she would never get pregnant again, or if she did she would continue to miscarry. After reading on the internet that she was at five times greater risk for recurrent miscarriages, she was understandably distraught. So we talked about where the number came from, and that while her risk may have increased, it was 20 times more likely she had just gotten unlucky than she had an underlying condition. (Explanation below if you’re interested.) Throughout the subsequent months and her following pregnancy we sent many long nights emailing each other back and forth, breaking down the scary statistics and rephrasing them in a more positive way.
That is my goal with the Miscarriage Estimator, to take a scary statistic and make it less so. 20% of pregnancies may result in miscarriage also means 80% of pregnancies are carried to term. Knowing doesn’t have to be scary.
Breaking down the “five times greater chance of recurrent miscarriage” statistic
The rate of miscarriage is generally assumed to be 20%. The rate of recurrent miscarriage is about 1%. That means of a group of 100 newly pregnant women we might expect 99 to have no underlying issues and 1 to have a as yet unknown recurring miscarriage condition. Of the 99 women without issue, roughly 20 will be unlucky and miscarry. The one with the recurrent miscarriage will also likely miscarry, for a total of 21 miscarriages out of 100. A women with a single miscarriage don’t yet know if she is the one with the recurrent miscarriage condition. The odds went from being 1 in 100 to 1 in 21, roughly about 5 times higher. Yet, while the odds are higher, she’s still 20 times more likely to not suffer from recurrent miscarriages.
Or perhaps, more accurately the title of this post should be “Six Months Out”. I left Google six months ago this past Sunday. At the time I was focusing on the move. I didn’t start working on my business until June, and didn’t launch my first new app since becoming an entrepreneur until July. Since I don’t know the exact date I started working on my business, I’ll have to use my departure date from Google as a mile marker.
In the past six months I’ve launched five Apps:
- Baby Name Uniqueness Analyzer
- Unique Baby Name Generator
- Photo Analyzer and Comparator
- Difficult & Extraneous Word Finder
- Passive Voice Detector (Now with More Zombies!)
The last three were launched silently.
I tried the new apps to the relevant old ones to increase their visibility. I have a link to the Difficult & Extraneous Word Finder & Passive Voice Detector from the Reading Ease Analyzer, and a link to the two baby name apps from the Labor Predictor. About 7.1% of my web traffic fallowed the links. I also had a 2% increase in traffic from people just visiting those new apps, which means they’re slowly being adopted, the baby name apps being the most popular.
One of my big lesson learns is to make it absolutely crystal clear what the app actually does. Despite having 42 visitors in the past month, not a single person has up loaded a file to the Photo Analyzer. Not a single one. I am thinking about setting up a tutorial for it. I applied this lesson to the Passive Voice Detector by giving it a sample text.
Sadly the incredible increase revenue from my ad placement fix did not last. Since then revenue has settled around a 77% increase. My guess is the giant increase was part of a newness effect, where users are attracted to and interact with new things. The ad placement is no longer new, and no longer as captivating to users.
I’m happy with the progressive I’ve made. I went from taking a week to write the Name Uniqueness Analyzer to less than a day to write the Passive Voice Detector. On the other hand, in six months I’ve only created five apps, and had a 77% revenue increase. I need a revenue increase of about 215 times (the equivalent of ten 77% increase) to justify continuing to work for myself long term.
Time to get back to work.
This year was Nicole’s first as a trick-or-treater, and our first in neighborhood with young children. Our town home was gated and secluded, the apartment complex full of recent college grads starting careers. We received one trick-or-treater back in 2012 or 2011. That was it.
Pausing only to snack on a piece of candy.
As much as it was Nicole’s first Halloween, it felt a bit like our first Halloween too. We were all excited. She counted down the days on her fingers. We bought another bag of candy every few days. For the trick-or-treaters, obviously.
Our big day went well, but there were a couple things we can improve upon next year.
Our Timing Was Way Off
Domingo and I weren’t sure when to start our trick or treating adventure. I remember spotting trick-or-treaters on my way to pick her up from daycare last year. That would have meant they were out around 5:30. Domingo was sure our neighborhood wouldn’t start until after six, as most of our neighbors don’t get home from work until then. (The fact that it was a Saturday slipped both our minds.)
We opted to wait until the first trick-or-treater stopped by our house: 6:20. We were in our costumes and out the door five minutes later.
By 7:00 all the “big kids” were out. Big kids as defined as bigger than Nicole. Normally Nicole is shy around other kids. That is, until candy is on the line. Still, she’d wait her turn standing next to the stoop while one group of kids would get their candy, only to be over crowded by anther group coming up the walk way. When we encountered our first group of highschoolers I decided it was time to turn in.
Next year we’re starting earlier. If some of our neighbors aren’t ready yet, that’s OK. They’re will be plenty of other houses to visit.
We had too much candy
Our home is on a Cul-de-sac on a hill. The slope necessitates the housing being a little more spread apart. The street below connects several circles of tightly packed houses on level ground. Our street is not prime picking. Not for an enterprising young kid who wants to maximize candy acquisition while minimizing energy expenditure. While on our street we encountered four other kids. Domingo handed out candy to 34 kids all night. Nicole and I easily encounter twice that during the brief time we were on the street below.
Next year we’re buying less candy. Or telling the kids to take more. One or the other.
The sun had set 20 minutes before we left our house and the light was already mostly gone. I was planning on bringing my camera with us, but as soon as I stepped outside I knew it would be for naught. I took 1 way under exposed photo, and 2 extremely bury of Nicole trick or treating at our own house, then left my camera at home.
But No Photos doesn’t always mean No Photos.
This afternoon we dressed Nicole up back into her costume and went outside to play. I ended up getting some great photos during the golden hour, a happy accident thanks to Daylight Savings Time. Since it was after Halloween, I didn’t care if she ruined her costume having fun. Basically it was ‘trash the dress’, Halloween toddler style.
Next year we’re also taking photos. Duh.
Happy Halloween from my monsters to yours!
My little monster love, love, loved her last year’s purple monster. She got a ton of extended us out of it with pretend play, so it was fresh in her mind when we started looking at costumes this year. When she saw the green monster she was hooked.
I wasn’t going to get Alexis a second costume this year, as she’s too young to go trick or treating. Then I was in Walmart and found an adorable baby monster costume and couldn’t resist the photo op.
I was afraid she’d hate the headpiece, but she didn’t seem to mind it much at all.
Nicole’s Costume – $16.15, or $0 depending on how you think about it. It’s her Trick or treat costume. (Amazon)
Alexis’ Costume – $15.97 (Walmart)
Total Cost – $32.12
Pretty much as soon as I hit “post” on your last month letter you completely lost interest in crawling. Standing and walking was where it was at. You wanted to stand all.the.time. It didn’t matter that your little legs weren’t quite strong enough, you wanted to be vertical and you wanted to be moving.
That weekend we got your first pair of shoes. We were sure you would need a bigger size than your sister. We were sure you had gotten Mommy & Daddy’s latent tall genes. Your feet seemed so long at birth. I used to put them in the palm of my hand, heal to edge. Your toes would dangle off the other side. I remember Nicki’s weren’t quite as long to reach the other edge. You were in the 60th percentile for height, a full 20 percentiles above your sister. You’re first shoe size? Another petite two. And just like for your sister, they seem too big!
You weren’t quite sure what to think of your shoes the first time. You’d tug and kick and get one off as soon as I’d get the other on. But once you understood what the shoes were for, you were hooked. The second time I got out your shoes you did your happy baby dance and couldn’t wait to go outside. You will be walking on your own soon, I’m sure.
But then, just as suddenly as you fell out of love with crawling, you began to pick it up again. This week you are all over the place. You love a good game of chase, both as the chasee and the chaser. I can tell we’re going to have a hard time keeping up with you!
Mommy & Daddy
It turns out that we are a two costume family, and not because we have two children. Each child apparently gets two costumes.
Alexis as Mini Mouse (her daycare costume)
It all started two years ago when Nicki was super girl. Her not-costume, costume was a big hit and school. Since it was basically just a dress, it was convenient for diaper changes. It also lacked some of the bulk of traditional toddler costumes, making it easy to play in without getting uncomfortably hot. Her teachers loved the practically of it. I loved that my child was happy all day.
The following year, Nicki was a monster. The week before her class’s costume party I began to fret about her losing the fur cuffs at school, or ripping the tulle skirt. I loved her monster costume, but it just didn’t seem practical for a day of playing at school. I opted to also get her a pair of cat ears and tail to pair with a black shirt and pants for the party instead.
The costume that’s comfortable for all day play is not necessarily the costume that’s comfortable when trick or treating at night. I’ve come to accept that fact, and that we are a two costume family.
Still, the thought of being one of those parents who spends $100 on costumes gives my inner cheap skate heart palpitations. I’m on the hunt for good, cheap costumes which can double for pretend play after Halloween. I want to get the most bang for my buck.
So far most of my daycare costume finds have come from the kid’s clothing section at Target. That’s where I got Alexis’ mini mouse dress, and Nicole’s super girl dress (Not quite the same one, but very similar.) They hold up well enough that Alexis was able to re-wear the super girl dress for Super Hero Day at school. Target seems to always have those not-quite-costume clothes, especially the super hero and frozen variety. I’ve also seen some at Babies R Us, but we don’t have one close by so I don’t shop there as often.
I’m also a huge fan of the costume accessories. The cat ears and tail cost under $5. (We got a similar bunny set for pretend play which can double as a daycare costume if we need it.) Add a $1 mask and now she’s a cat burger. Or pair the mask with a tutu and cape (already owned) and now she’s a crime fighter. Every year we check out the accessories to see if there’s anything good for pretend play.
The other place I like to shop is online. Once Nicole picked out her costume, I used the reverse image search on the stock photo. Most retailers use the same the stock photo created by the manufacture. By doing a reverse image search on the stock photo I can find all the websites where the photo is shown, i.e. all the retailers selling the same item. From there it’s a simple process to find the one with the best price.
This year I spent $33.12 in total on Nicole’s costumes, and $34.97 on Alexis’. That’s only $5-7 up from the $27.85 average, and each girl get’s two! I’ll take that!
Mini Mouse Dress – $18 (Target)
Bow – $1 (Walmart)
Backdrop – $12 (JoAnns – not purchased for this photo session, but I thought I should include it anyway)
Total cost – $31
Last Saturday I small made a change to my webapps, one that resulted in a 164% increase in revenue. The difference? Slightly different ad placements. I used to place ads at the bottom of the page, below tool descriptions near the terms of service. Now they’re right below the tools. That’s one of the reasons I’ve always liked data analysis and human computer interaction. Sometimes the smallest differences can have profound affects.
We’ve been in a bit of a rough patch sleep wise this week, and my creative juices have been a bit zapped. Rather than working on new apps I spent some time cleaning up my code base. I’m a fan of rapid prototyping, but sometimes rapid prototyping leads to sloppy prototyping. I moved common code to a few custom libraries, which should make it faster to stand up new apps (when the creativity returns.) I’m not one to let ugly code stand in the way of progress, but I sure do prefer pretty code.
One of the decisions I’m struggling with is what to do with the two versions of the labor probability calculator.
Logically I should pitch the new version. The new version has been live for a very long time. (A couple years maybe? I can’t remember, I blame mom brain). When the new version launched, the old version was accessible only via a parameterized link. Meaning if one bookmarked the calculator before the calculator before the new version launched, they would still see the new version by default. One would have to actively select the old version. So it’s pretty significant that today 60% of the web traffic is for the old version. My user base is preferring the old version, and I’m not sure why. The old version is also x8 more profitable than the new version for the silly reason that it involves page refreshes. If it’s more popular and more profitable, it should be the default version, right?
My users may prefer the old version, but I like the new. Fundamentally, they’re identical; they both calculate the same odds, and have the same default dates. The only difference is the graph in the new version – something that very much appeals to my inner math geek. I think I’m going to take the anti-data drive approach and leave the two versions as is. At least until I can figure out what it is about the old version that makes it so much more attractive. Please don’t be the lack of a graph.