Magento Developers Paradise 2010 – Mallorca, Spain

In our commitment to specialising in Magento Hosting, Create Hosting recently attended the Magento Developers Conference which was held over 4 days (Oct 9 – 12 2010) in Mallorca, Spain.

This was a fantastic event organised by the good guys at Netresearch (Annemarie, Thomas and the rest of the team) and further supported by some great sponsors such as PayPal, Visions New Media, Cash-Ticket and AheadWorks to name a few.

The event was a great success, with a chance to meet with the Magento team and over 140 other Magento developers and experts from all backgrounds, to discuss and share ideas. There is strong community involvement amongst Magento developers, more so than I have seen with any other open source platform. There was an overriding theme at the conference on providing fine tuned and well engineered Magento solutions, and great discussions on how to scale Magento within the hosting environment, application and coding levels.

Here’s a highlight of some of the sessions:

  • Keynote
  • Magento for PHP MVC Developers
  • Getting the Most of Magento Themes
  • Top Tips for Developing a Successful Magento Extension
  • Magento Connect
  • Zend_Search_Lucene
  • PayPal’s Wallet in the Cloud
  • PayPal Integration in Magento
  • Methods and Best Practices for High Performance Magento
  • Clone Yourself
  • Making Magento Run Like an Antelope

Keynote

Yoav Kutner (Magento CTO)

Yoav gave us a “state of the nation” for Magento and some insight into where Magento is heading. This was quite an extensive session, covering many areas but I’ll keep it brief.

To start, some stats:

  • Magento downloaded over 2.5 million times
  • Searches for the term “Magento” outstrip searches for the term “Ecommerce”
  • Over 65K (estimated) active merchants using Magento across the Globe (according to Classy Lama Studios, this is calculated by the number of Magento admin notification updates/calls)
  • Over 2.4 miliion extension downloads
  • Over $25 Billion (estimated) in transactions, processed annually through Magento stores
  • Over 180 solution partners

And a roadmap for 2010/2011:

  •  Strong Community Plan to monitor, grow, foster and reward the online community (glad to see this acknowledgment)
  • Magento Mobile (launched) for Apple, Android and Windows
  • Magento Professional – a Mid Tier, enhanced community edition (launched). Cost is US$2995/yr. Available via solution partners.
  • The Magento Community Edition will have one more release before end of the year. Version 1.4.1.2 is due out November 2010 and will be a maintenance release that should fix 450+ bugs.
  • This version will include lots of optimisation in sales!
  • Magento Community Edition 1.5 (tentative)
    • Ordering composite products in the Admin
    • Edit order without creating a new one
    • Custom order statuses
    • Multistep checkout with no JavaScript
    • Improved Import Functionality, with support for Simple, Virtual, Grouped, Configurable and Gift Card products. It will also support all types of attributes, different values per scope, tiered pricing (thanks Magento) and custom options. Note, the uRapidFlow commercial extension already offers all this and more.
  •    Magento Community Edition 2.0. This version is being built from the ground up, with improved architecture and support for advanced features – have your say at http://bit.ly/azvQFT. Goals include:
    • Improve product quality: unit tests, automated UI tests, continuous integration process (bamboo)
    • Improve product performance and scalability: load tests, data scaling, media scaling (CDN)
    • Improve product security: client and server side data validation everywhere
    • Modular architecture: moving framework related code on library level, reorganizing view level, modules decoupling (remove Mage class)
    • 100% coverage of Web Services API
    • Multiple (R)DBMS
    • Improve multiple languages support (maybe remove dependency of lang and store views)

Magento for PHP MVC Developers

Alan Storm (eCommerce Blogger and Magento Developer)

What can I say except that this session gave me an “aha moment” and now I understand [a little more]. Alan Storm is brilliant and contributes greatly to the Magento community. Read Alan’s blog at http://alanstorm.com.

Getting the Most of Magento Themes

Anton Makarenko (Magento Technical Project Manager)

Gained a greater understanding of the theme hierarchy and the new fallback mechanism. If creating your own theme, use the base/default theme. Be careful with using “removes” as this still consumes memory.

Top Tips for Developing a Successful Magento Extension

Ashley Schroder (Head Software Engineer, World Wide Access)

Ashley gave us a great talk on the reasons for using extensions as a way of developing Magento, the pitfalls to look out for and some ways of managing your extensions effectively. Ashley also gave us some ideas for backing up your Magento to the cloud, which we’ll be researching and providing for Create Hosting Magento customers in the very near future.

Zend_Search_Lucene

Stephan Hoyer (Senior Developer, NetResearch)

Stephan gave us a brilliant overview of the open source search module Zend_Search_Lucene, an advanced search engine for Magento. While advanced search is available for Magento Enterprise, there is a gap in the market for a decent search for Magento Community Edition. Features include:

  • multi tier search
  • layered navigation
  • index categories, pages (including static blocks)
  • prohibit words
  • API to search for other content, such as blog posts

If you’re interested, join Netresearch and Zend in a joint webinar on Zend_Search_Luceneon November 30th 2010.
http://www.zend.com/en/company/news/event/webinar-search-in-the-magento-shop-better-with-zend-search-lucene

PayPal’s Wallet in the Cloud

John Lunn (Head Wizard, PayPal X)

John gave us a brilliant session on “how to get your wallet in the cloud”. An alternative to cash. PayPal X will be a fast mover in the arena. Don’t ignore it.

  • Think about how people will access your site.
  • Use Magento Mobile functionality – ASAP.
  • Think about how Magento could be used in the real world.
  • Why build systems that you need to login to? Use someone else instead (oAuth, OpenID, etc).
  • Think outside the box – the world is changing faster than ever!

PayPal Integration in Magento

Anton Makarenko (Magento Technical Project Manager)

Magento have made some serious progress with PayPal, making it a lot easier for customers and developers. Improvements in customer experience:

  • Configurable logos and buttons
  • Custom payment selection
  • Cart line items transferring
  • EC shipping options transferring

Plus,

  • Settlement Reports Fetching
  • Billing Agreements
  • Recurring Profiles
  • Cart line items transferring
  • EC shipping options transferring
  • Improved business logic
PayPal – Country Specific

PayPal – Country Specific

Magento Connect

Jonathan Beri (Senior Technical Product Manager, Magento)

Jonathan gave us a great presentation on the extensions – for today and beyond; a roadmap for Magento Connect and discussion with developers on how Magento Connect can be improved.

Jonathan also covered Magento Connect 2.0 concepts, highlighting some of the following features:

  • Recommendations
  • Compatibility reviews
  • Analytics
  • Promotional opportunities
  • Subscription (instead of one-time purchase)
  • Encoding Extensions
  • New opportunities, e.g. Mobile
Current extension map – opportunity

Current extension map – opportunity

Magento Connect 2.0 – Concept

Magento Connect 2.0 – Concept

Methods and Best Practices for High Performance Magento

Dmitry Soroka (Magento Lead Architect)

Dmity gave us an in-depth coverage of how to optimise Magento at the environment and application level, as well as best practices on scaling Magento for high transactions. These were all backed up with benchmarks. Topics covered:

Environment Optimisation

  • Web Server Configuration (keep Apache modules to a minimum, KeepAlive directive,
  • using mod_deflate, eliminating directory structure scans for the .htaccess by moving to vhost.conf, etc)
  • Data Base Configuration (innodb_buffer_pool_size, innodb_thread_concurrency, table_cache, query_cache_size, etc)
  • Accelerating PHP (APC, eAccelerator, Zend Optimize, Xcache, tmpfs memory file system, compilation)
  • Additional HTTP Requests (combining CSS, Javscript, images / sprites

Application Optimisation

  • Caching (cache types, cache storage, two levels backend)
  • Session Handling
  • Additional options (flat catalog, sales archive, backend processes (cron, indexes) & theme complexity

Scaling

Determine the maximum amount of concurrent users that will hit your server. Having a separate backend server (database, handling admin users, processing backend activity with cron, pre generating full page cache, handling media queries). Ensuring good connection between master and slave.

Making Magento Run Like an Antelope

Philippe Humeau (Director, NBS System)

NBS System are the leader in Magento hosting in France, so I was particularly interested in what Philippe had to say. The focus was on reducing page load times, so this session aligns nicely with our goals at Create Hosting. While the information is more applicable for high transactional websites with 100,000+ products, my view is that the concepts still apply – especially when Magento in a global multi-store type setup. Or with offering specialist Magento hosting in multi-tenanted / shared hosting environment.

Recently Google have made it very clear that they expect a 1.5s page load time. If you can meet this target it will give you a strong advantage in SEO. Within a few months you need to be there!

Hardware & OS

  • Intel XEON Dual Quad Core (e.g. 5670) for frontend, AMD mono – hexacores for DB
  • DDR3 (at least 2GB / core)
  • SAS controllers with cache
  • Windows consume too much ressources on PHP interpretation, weak TCP/IP stack, not optimized for *AMP environment
  • OpenSolaris gave poor results (30% less than Linux)
  • Debian Lenny, CentOS, NetBSD or Redhat (avoid the GUI based distro)

An Efficient Architecture

  • Firewall & load balancer
  • Setup a reverse proxy to offload the server (NGINX, Squid) – 20-40% performance increase
  • Separate web & db servers – 33% performance increase
  • Content server for static files (CSS/JPG/GIF/JS…)
  • Deliver static content near to the user with CDN (Content Delivery Network)

Coding / Application

  • Coding matters – a lot! A small glitch in a loop can take down the server, especially when amplified by lots of visitors
  • Use extensions like Fooman Speedster, Magento boosters (like delorum’s one)
  • Read forums and blog posts and apply the optimisation tips
  • Get consulting from Magento Inc, and save yourself a lot of time!
  • Go to a strong training course
  • Understand the block caching
  • Take care of security issues! (xss/csrf/sql-injection)
  • Don’t leave a 404 error on a picture or create a PHP exec to output some content!
  • Optimise the CSS selectors & the CSS calls (just need one)
  • Serialize CSS and then JS order in the downloads (not css/js/css/js)
  • Minify your JS using Jsmin
  • Avoid redirections, optimize pictures, and DNS queries

Breaking the 1.5 Second Barrier

  • Reduce I/O bottlenecks (RAID 10, cache and session on tmpfs RAM disks, don’t log everything)
  • MySQL – Split read & write to different DB:
  • -> host it on a RAMDISK
  • -> Host it on a Physical Disk
  • Some Advanced Tweaking
  • Use Sysfence to autokill the bombing crons or php process
  • Truncate (or archive) your log tables in Magento
  • Check foreign includes response times and put them at the end of pages
  • Use pagespeed / Yslow / Gtmetrix to analyse your site
  • Integrate Lucene SOLR on large catalog database (> 100 000 products)
  • Use latest version of Magento and activate all caches in the backend (if possible)

more…

  • Analyze your traffic (throttle mailouts, same for all traffic generation – burst is your enemy)
  • Use cache the right way
  • PHTML blocks should be cached it in order not to be generated each view
  • To cache a block, you have to declare it explicitly
  • Developers usually don’t use the Block caching mechanism at all (80%)
  • Up to ten times faster loading when using cache properly!

Wrap Up

These last two sessions were invaluable, and the information covered will assist us in scaling our Magento hosting options to grow with the needs of our customers. Keep an eye out for more posts on Magento optimisation. We’ll be doing our own R&D, both at the environment and application level. We’ll be providing extensive coverage of application optimisation, covering a full gamut of available techniques and providing benchmarks for each.

Some Useful Community Resources:

http://www.magentocommerce.com/
http://groups.google.com/group/magento-devel
http://twitter.com/ask_magento – for questions and answers that take up less than 140 characters
http://twitter.com/magento_ce_svn – preview releases
http://twitter.com/magentoconnect – Magento Connect
http://twitter.com/magentomobile – Magento Mobile
http://twitter.com/magento – Main Magento account
http://www.linkedin.com/company/magento
http://www.youtube.com/magentocommerce
http://www.slideshare.net/varien

Other Blog Posts:

Sand & Source Code – Magento Developer Paradise, 2010 (Jonathan Beri)
Magento Developers Paradise, Day One (Mark Hopwood)
Magento Developers Paradise by the Numbers (Ashley Shroder)
Magento Developers Paradise Summary (Classy Llama Studios)
Tsvetan Stoychev’s Personal Blog

Plus, out and about in Majorca

Netresearch organised some excursions (Hummers, Jeep Wranglers and Nassau Beach Club). Hop in a rental car and there’s a beautiful island to explore.