At the begining when I and my friends started working on our startup we tried to design, develop and ship a complete product. At some point we decided that we’re running out of time and pushed out what we had – crooked, sloppy version of what we wanted to ship. Even though a product that was supposed to be a mailbox was failing when sending emails it turns out it was a very good decision.
At that time mail4group was only available in Polish and its main target were Polish students. The academic year of 2011 was about to start. We pushed the incomplete product with some features working, some not working, some breaking in the middle and emailed it to some of our friends. We quickly got around 1000 of first users, that were quite happy with basic functionality but often encountered an error page. In fact, it was so common that we even gave it a fancier design with a ninja running through the screen. Ninja soon became an official, front page mascot of the portal.
Each error page would generate an email and send it to all founders. Sometimes we were flooded with ~1000 error emails a day, but this was a good amount of testing we would never get ourselves. What is more, we were able to notice which paths users follow so we were able to quickly removed completely unused features and enhance those that were used. Google analytics was and still is a most important part of knowing which features to develop.
A year after that we met with our friends from Brainly.com, and the main point of discussion was that we don’t measure enough. Even though we attempted to measure each of our main features and analyze the outcome, in their eyes we were still novices in measuring what we do. At that time attaching events or A/B experiments to every single change that we do sounded extreme and in the clouds, but soon, as we moved forward we began to appreciate structured metrics.
We once changed a registration form to include a beautiful, cheerful background from its original gray and sad one that we currently had. I liked it, my friends liked it, my sister liked it. And it dropped our conversion rate by 4%. Why? I still am not sure, maybe too vivid background turned the attention away from the actual registration form. Maybe people thought that they are not as dynamic as people in the background photo and should find something elegant. Point is, if it wasn’t for A/B test for a simple background change we would end up doing the wrong thing.
The very same registration form which in our case was special, because it involved creating both a group name and individual account faced many more A/B tests during our journey. We made tests deciding whether initially fields should be hidden, or how much data people will give us without decreasing conversion rate. We ended up chopping it down to 3 fields and asking for everything else after use was initially registered. But at that time it was educated choice, we had data to tell us that if we do this, we’ll get 30% less people abandoning registration.
The lesson I learned from Brainly was that the smallest shippable product can be as small as not working button. Having a good button with an interesting label may allow you to actually count how many people clicks it. And one you know it, you can decide whether to build the function or not. This was actually a very good piece of advice for Ewa, who asked me about this part. She had a portfolio page with photos of her designer furniture and some other stuff. She wanted to start selling it on her website. Instead of integrating an online shop sollution to know if anyone actually buys it there she could have just place a button there, attach a google analitics event there and open email window with pre-populated item name. One hour work instead of probably a week, and instantly she could start monitoring interest in buying her items. Plus you don’t risk wasting money and effort on a solution that’s not giving you enough profit.
Part two will cover:
Metrics, Pivoting and Lean startup