Archive for August, 2013
7 things I’ve learnt in 7 months at 7digital
Posted by Michael Okarimia in 7digital, Books on August 18, 2013
In January this year I started working at 7Digital, the leading global open music platform, in the Search & Content Discovery Team.
My role as a developer was quite a change from my previous jobs and I have relished the challenge of something new.
I’ve learnt not only new technologies but also better methodologies. Here are the 7:
1. Monitoring
Real time application monitoring with New Relic. Install the New Relic agent upon the server you wish to monitor, and with a little configuring, your web app will now start logging a wealth of metrics such as web request and response times, database query time and error rates. The visualisation of this data is very slick too.
StatsD & Graphite, which allow monitoring of any custom metric and presenting it in easy to read graphs. 7digital take a similar philosophy to Etsy, and favour monitoring production applications over trawling through log files. StatsD has a C# client library which we make extensive use of. Written by fellow 7digital developer, Goncalo Pereira it allows us to monitor any metric by sending UDP packets to a StatsD server. Unlike New Relic, metrics must be explicitly defined within the application by writing code that calls the library. This lets us monitor the areas that New Relic doesn’t, along with monitoring very specific events. Being able to see Zabbix stats is often a benefit too.
2. Continuous Delivery
TeamCity and continuous delivery, which makes deploying to production a simple case of clicking on a few buttons on the Team City project page, which allows 7digital to deploy to live dozens of times a day. Coupled with the extensive monitoring that we have, live deploys are low drama affairs.
3. Linux
Use of Raspberry Pis for displaying the Graphite monitoring graphs encouraged me to have a tinker with a Pi at home too. 7digital now use Pis for monitoring of all their production statistics, which are affixed to monitors hanging from the ceiling around the office.
I now work with Linux servers in production. I’d played with Linux in my own time at home, but now that our team is responsible for ensuring performance of the Solr infrastructure, I’ve learnt an awful lot.
4. Solr
Solr is an HTTP accessible search platform powered by Lucene. The 7digital’s search and catalogue API endpoints are powered by Solr servers. A large part of my day to day activities relate to improving the search performance in the 7digital API, understanding how best to insert documents into the Lucene index and understanding what makes a good schema. My colleague James Atherton has given a presentation to the SolrLucene Revolution conference in San Diego earlier this year, discussing what has been learnt in the process of migrating from a SQL Server text search system to a scalable Solr infrastructure system. As part of managing the infrastructure, I’ve just begun to learn about configuration management using CFEngine to manage server configuration in an automated fashion.
5. RESTful APIs
7digital’s API a RESTful one. Each endpoint on the public API routes to an internal API which is owned by a development team. OpenRasta is the web framework used in my team, which was quite different from working with ASP.Net MVC powered websites.
6. Methodologies inspired by The Goal, and The Phoenix Project
The Goal: A Process of Ongoing Improvement and The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win
are books which everyone at 7digital has been encouraged to read. Both of which are novels concerning organisations that are struggling to improve their engineering processes, and succeed once they focus on finding the bottlenecks in the processes that are slowing them down. The Goal was originally written in the 1980’s and focussed on a manufacturing plant’s processes, which correlate surprisingly well with software delivery processes. The Phoenix Project is essentially a modern rewrite of The Goal, but focussed on an IT department within a larger organisation. Both books vividly describe the problems many developers, Sys Admins and managers have experienced first-hand. I liken it to Clean Code but for organisations.
7. Team
Working with a smart team who can deliver results is a great experience. Compared my previous workplaces there’s a more collaborative approach to problem solving across the teams. The levels of expertise within the company is inspiring and I always learn something new each time I informally chat with a member of another team.
I’ve been impressed so far and I look forward to what the future projects will bring!