After a long time I atttended well concieved event.
[updated : 11th July 2011] : Urls of site/people/projects
[updated : 11th July 2011] : Renaming and few corrections/disclaimer.
This is not complete view as I could not attend other fabulous sessions around membase, memcache,choosing right php framework and gearman. But I did hear great things about them, hoping to catch them on videos.
1. Event website, content, payment process, organizers
Conceptualization of event was validated with potential speakers, their talks and votes of interest. Kiran took pains to look at public profile and github of speakers. A very nicely designed website for potential attendees was the first temptation – really a blowback to vendor backed agency cookie cutter output. Although not all talks were focused on php or deep php , the theme connected cloud hosted php website.
Smooth checkout/payment integration was another plus. Wished there was a button, come later details will be preserved, as I struggled to justify staying out for full day away from little kiddo and other stuff as i entered details multiple times.
Another good thing was no over hype of changing the world kind of thing. A bunch of professionals who wanted to provide a reasonable venue for exchange of thoughts. More importantly no marketing only types or speaking and no work types projecting themselves as gift to humanity. This was apparent in most of the speakers especially zynga, facebook, capillary, mobstac & some yahoo folks.(after the event impression meeting/exceeding the expectation).
2. Event venue – Dharmaram college inside christ college with good parking and plenty of trees was idyllic setup. College had really good quality projectors in place. Food, tea, biscuits were good- how can I say anything bad about food which provides pineapple gojju . Restrooms were plenty and clean. Water was available when needed. Only challenge was Internet, which I personally thought was a good riddance. Then there was event photographer Kushal Das omnipresent with his gadgetry.
3. Content & people
My expectations were around learning best practices across the technologies. It was wonderful to see pythonistas around prepping for pycon. There were lot of offline discussions around chef, puppet, ganglia, capstriano. I saw folks coding up in both django and a template I could not make out plus pure lamp stack or just plain cgi.
Flipkart continous deployment
I personally liked flipkart’s sessions – real people sharing real insights, best practices with humility previously not seen elsewhere. Flipkart’s sharing of how they do continous deployment – major 6 + releases everyday – I thought I saw 30 , although speaker could have done with double shot of coffee , a person who knows his stuff but sort of shy in speaking up. There are people with less than ounce of his depth who create unfortunately much perception of depth – they can talk without single shipping application.
Anyway coming back to content – main take away was use of debian packages to do deployment. They provision bunch of new machines after netboot to deploy right os, puppet to do configuration. Then finally debian packages are used to deploy the code base, dependencies etc. Easy to roll back, audit. Some times somethings are still done by hand(schema change). They can do rolling deployments most of the times. They do use very heavy cache and solr for search. This sessiom focussed on how every commit results in build, deployment. They replicate as much of production environment into dev, something we push many customers at least for db size.
45 minutes were too short into get details of what tests they run, how they do perf testing, how they profile, use monitoring ( gomez/keynote) vs pingdom etc). Or dig into usage of logs to raise alerts, or dynamic monitoring of resource usage across php, jvm, nginx, ha proxy or varnish, network components. As they shared later in another session they have home made load balance running linux in ha mode with multiple network cards to allow 3 networks down.
Amol Amod (thanks Mekin) clarified lot of questions wrt deployment, again very humble down to earth folks.
Flipkart performance tips
This talk by Siddarth was worth its time in Gold. Flipkart uses php minus apache front ended by ha proxy, varnish and nginx. I am assuming they use memcache. Explanation of various options and why they chose what they had was well detailed.
I think/assume usage of solr takes off whole lot of load and thrifted services are invoked only for real work with db. Since they do not store any state anywhere, near horizontal scale is possible at web, services layer. They have interesting component in java as threaded service/deamon which runs locally on every web server. This component is the one which does async, parallel calls, does logging etc. Imho this is really the guts and choke point of app and I really could not get the point of this abstraction as they could talk from php to thrift service(except parallel calls). Availability of this component, failover was something time did not permit to get into. Caching, performance tools across proxy,cache(static-dynamic), web server, php, jvm(any particular gc setting?), solr,cache, db was not delved in deep as time was less. It would would been awesome to get into discussions around why of templates or no templates, models in php vs abstracted behind service layer. Similar discussion around use of jquery vs other frameworks, usage of cdn would have been great. One thing I did not see was use of any queue or analytics(not really part of the topic so okay) or nosql, functional lang ( was expecting some scala vs clojure adoption) discussion. Very-2 pragmatic and generous. Almost too generous for sharing pretty much their core. Even today it is difficult to find how amazon does its website scaling inspite all talk of openness etc. Flipkart’s another good thing was that they did not boast because they use a “tech/lang”.
Practo and usage of beanstalkd
This particular talk by practo co-founder Abhinav lal went into how he made the choice in favor of beanstalkd over rabbitmq or expensive message buses. He rightly called his usage as job queue where he pushes priority based messages into tube and has workers to process those messages. He uses ttr and persistence ( not clear whether it is automatically purged after some time) to pull messages and process. Usage of supervisord to scale out, monitor the worker process was a neat trick he shared. Looked like he used php on worker as well as client side so serialization of data was no big deal. Overall beanstalkd came out as decent job queue and excelling at it. There is no fancy publish/subscribe, esb, jms infra on top of it. Again a great speaker. I did bump into beanstalkd via delayedjob earlier. So good validation of “scale” vs “usage”. Again a great value for money discussion.
Failure as an option by Vijay from zynga
This was the talk delivered by a person who had guts to say things as they are. Specifically around cloud – unbound resources, elasticity, provisioning etc….basically burst lot of myths… importantly he had imp advice – it is not possible to know/control all unknowns. So at least control your destiny where you can. He adviced to have few reserved or passive instances to overcome the tyranny of ‘avalanche ‘ when cloud provider faces downtime, all clients try to provision more resources. He added other ideas around why/how to failover completely for x userbase vs half baked response to everybody. Now this requires lot of work and thinking at lot of layers – it would be ideal to apologize only to people whose service went down rather than fail whale for everybody. His suggestion of usage of ha proxy intelligently, not failing over all load of 5 machines to remaining 5 ( out of 10)if those 5 machines are at 50% + usage already. This would start cycle of provisioning and more load on already stressed infra.
Ravi pratap‘s talk turned out to be ec101 rather than being true to title of scaling. He I am sure could have done justice to that content too as he is great speaker with lot of hands on work. His explanations were crisp and he shared one imp thing thing- they have their own ami as expected. Again time was too short for him to get into details of scaling as covering basics took longer time. But would expect at some other location he can share his tips.
Eucalyptus talk was big let down unfortunately as it would have been easier to start of with demo rather than explain cloud again…certainly not cool to run down valid competition as malware.
Python tool fabric as deployment option by capillary (Nigel Babu) was completely new subject for me.
Thrift talk by capillary(Piyush Goel) on how they use it for communication from php to java service was good tutorial. Main adv over protocol buffers was lack of rpc in protobuf.
Again thanks to @hasgeek for conjuring up goood meet. http://phpcloud.hasgeek.in
Conducted on 9th july 2011