Create Hosting at Magento Developers Paradise – Mallorca, Spain

In our commitment to specialising in Magento Hosting, Create Hosting recently attended the Magento Developers Conference which was held over 4 days (Oct 9 – 12 2010) in Mallorca, Spain.

This was a fantastic event organised by the good guys at Netresearch (Annemarie, Thomas and the rest of the team) and further supported by some great sponsors such as PayPal, Visions New Media, Cash-Ticket and AheadWorks to name a few.

The event was a great success, with a chance to meet with the Magento team and over 140 other Magento developers and experts from all backgrounds, to discuss and share ideas. There is strong community involvement amongst Magento developers, more so than I have seen with any other open source platform. There was an overriding theme at the conference on providing fine tuned and well engineered Magento solutions, and great discussions on how to scale Magento within the hosting environment, application and coding levels.

Here’s a highlight of some of the sessions:

Sessions

Keynote
Magento for PHP MVC Developers
Getting the Most of Magento Themes
Top Tips for Developing a Successful Magento Extension
Magento Connect
Zend_Search_Lucene
PayPal’s Wallet in the Cloud
PayPal Integration in Magento
Methods and Best Practices for High Performance Magento
Clone Yourself
Making Magento Run Like an Antelope

Keynote

Yoav Kutner (Magento CTO)
Yoav gave us a “state of the nation” for Magento and some insight into where Magento is heading. This was quite an extensive session, covering many areas but I’ll keep it brief.

To start, some stats:

Magento downloaded over 2.5 million times
Searches for the term “Magento” outstrip searches for the term “Ecommerce”
Over 65K (estimated) active merchants using Magento across the Globe (according to Classy Lama Studios, this is calculated by the number of Magento admin notification updates/calls)
Over 2.4 miliion extension downloads
Over $25 Billion (estimated) in transactions, processed annually through Magento stores
Over 180 solution partners

And a roadmap for 2010/2011:

Strong Community Plan to monitor, grow, foster and reward the online community (glad to see this acknowledgment)
Magento Mobile (launched) for Apple, Android and Windows
Magento Professional – a Mid Tier, enhanced community edition (launched). Cost is US$2995/yr. Available via solution partners.
The Magento Community Edition will have one more release before end of the year. Version 1.4.1.2 is due out November 2010 and will be a maintenance release that should fix 450+ bugs. This version will include lots of optimisation in sales!
Magento Community Edition 1.5 (tentative)
- Ordering composite products in the Admin
- Edit order without creating a new one
- Custom order statuses
- Multistep checkout with no JavaScript
- Improved Import Functionality, with support for Simple, Virtual, Grouped, Configurable and Gift Card products. It will also support all types of attributes, different values per scope, tiered pricing (thanks Magento) and custom options. Note, the uRapidFlow commercial extension already offers all this and more.
Magento Community Edition 2.0
This version is being built from the ground up, with improved architecture and support for advanced features – have your say at http://bit.ly/azvQFT. Goals include:
- Improve product quality: unit tests, automated UI tests, continuous integration process (bamboo)
- Improve product quality: unit tests, automated UI tests, continuous integration process (bamboo)
- Improve product performance and scalability: load tests, data scaling, media scaling (CDN)
- Improve product security: client and server side data validation everywhere
- Modular architecture: moving framework related code on library level, reorganizing view level, modules decoupling (remove Mage class)
- 100% coverage of Web Services API
- Multiple (R)DBMS
- Improve multiple languages support (maybe remove dependency of lang and store views)

Magento for PHP MVC Developers

Alan Storm (eCommerce Blogger and Magento Developer)
What can I say except that this session gave me an “aha moment” and now I understand [a little more]. Alan Storm is brilliant and contributes his knowledge at http://www.magentocommerce.com/knowledge-base. Read Alan’s blog at http://alanstorm.com.

Getting the Most of Magento Themes

Anton Makarenko (Magento Technical Project Manager)
Gained a greater understanding of the theme hierarchy and the new fallback mechanism. If creating your own theme, use the base/default theme. Be careful with using “removes” as this still consumes memory.

Top Tips for Developing a Successful Magento Extension

Ashley Schroder (Head Software Engineer, World Wide Access)
Ashley gave us a great talk on the reasons for using extensions as a way of developing Magento, the pitfalls to look out for and some ways of managing your extensions effectively. Ashley also gave us some ideas for backing up your Magento to the cloud, which we’ll be researching and providing for Create Hosting Magento customers in the very near future.

Magento Connect

Jonathan Beri (Senior Technical Product Manager, Magento)
Jonathan gave us a great presentation on the extensions – for today and beyond; a roadmap for Magento Connect and discussion with developers on how Magento Connect can be improved.

Current extension map - opportunity

Jonathan also covered Magento Connect 2.0 concepts, highlighting some of the following features:

Recommendations
Compatibility reviews
Analytics
Promotional opportunities
Subscription (instead of one-time purchase)
Encoding Extensions
New opportunities, e.g. Mobile

Magento Connect 2.0 - Concept

Zend_Search_Lucene

Stephan Hoyer (Senior Developer, NetResearch)
Stephan gave us a brilliant overview of the open source search module Zend_Search_Lucene, an advanced search engine for Magento. While advanced search is available for Magento Enterprise, there is a gap in the market for a decent search for Magento Community Edition. Features include:

multi tier search
layered navigation
index categories, pages (including static blocks)
prohibit words
API to search for other content, such as blog posts

If you’re interested, join Netresearch and Zend in a joint webinar on Zend_Search_Luceneon November 30th 2010.
http://www.zend.com/en/company/news/event/webinar-search-in-the-magento-shop-better-with-zend-search-lucene

PayPal’s Wallet in the Cloud

John Lunn (Head Wizard, PayPal X)
John gave us a brilliant session on “how to get your wallet in the cloud”. An alternative to cash. PayPal X will be a fast mover in the arena. Don’t ignore it.

Think about how people will access your site.
Use Magento Mobile functionality – ASAP.
Think about how Magento could be used in the real world.
Why build systems that you need to login to? Use someone else instead (oAuth, OpenID, etc).
Think outside the box – the world is changing faster than ever!

PayPal Integration in Magento

Anton Makarenko (Magento Technical Project Manager)
Magento have made some serious progress with PayPal, making it a lot easier for customers and developers. Improvements in customer experience:

Configurable logos and buttons
Custom payment selection
Cart line items transferring
EC shipping options transferring

Plus,

Settlement Reports Fetching
Billing Agreements
Recurring Profiles
Cart line items transferring
EC shipping options transferring
Improved business logic

PayPal - Country Specific

Methods and Best Practices for High Performance Magento

Dmitry Soroka (Magento Lead Architect)
Dmity gave us an in-depth coverage of how to optimise Magento at the environment and application level, as well as best practices on scaling Magento for high transactions. These were all backed up with benchmarks. Topics covered:

Environment Optimisation

Web Server Configuration (keep Apache modules to a minimum, KeepAlive directive,
using mod_deflate, eliminating directory structure scans for the .htaccess by moving to vhost.conf, etc)
Data Base Configuration (innodb_buffer_pool_size, innodb_thread_concurrency, table_cache, query_cache_size, etc)
Accelerating PHP (APC, eAccelerator, Zend Optimize, Xcache, tmpfs memory file system, compilation)
Additional HTTP Requests (combining CSS, Javscript, images / sprites

Application Optimisation

Caching (cache types, cache storage, two levels backend)
Session Handling
Additional options (flat catalog, sales archive, backend processes (cron, indexes) & theme complexity

Scaling
Determine the maximum amount of concurrent users that will hit your server. Having a separate backend server (database, handling admin users, processing backend activity with cron, pre generating full page cache, handling media queries). Ensuring good connection between master and slave.

Making Magento Run Like an Antelope

Philippe Humeau (Director, NBS System)
NBS System are the leader in Magento hosting in France, so I was particularly interested in what Philippe had to say. The focus was on reducing page load times, so this session aligns nicely with our goals at Create Hosting. While the information is more applicable for high transactional websites with 100,000+ products, my view is that the concepts still apply – especially when Magento in a global multi-store type setup. Or with offering specialist Magento hosting in “shared hosting” environment.

Recently Google have made it very clear that they expect a 1.5s page load time. If you can meet this target it will give you a strong advantage in SEO. Within a few months you need to be there!

Hardware & OS

Intel XEON Dual Quad Core (e.g. 5670) for frontend, AMD mono – hexacores for DB
DDR3 (at least 2GB / core)
SAS controllers with cache
Windows consume too much ressources on PHP interpretation, weak TCP/IP stack, not optimized for *AMP environment
OpenSolaris gave poor results (30% less than Linux)
Debian Lenny, CentOS, NetBSD or Redhat (avoid the GUI based distro)

An Efficient Architecture

Firewall & load balancer
Setup a reverse proxy to offload the server (NGINX, Squid) – 20-40% performance increase
Separate web & db servers – 33% performance increase
Content server for static files (CSS/JPG/GIF/JS…)
Deliver static content near to the user with CDN (Content Delivery Network)

Coding / Application

Coding matters – a lot! A small glitch in a loop can take down the server, especially when amplified by lots of visitors
Use extensions like Fooman Speedster, Magento boosters (like delorum’s one)
Read forums and blog posts and apply the optimisation tips
Get consulting from Magento Inc, and save yourself a lot of time!
Go to a strong training course
Understand the block caching
Take care of security issues! (xss/csrf/sql-injection)
Don’t leave a 404 error on a picture or create a PHP exec to output some content!
Optimise the CSS selectors & the CSS calls (just need one)
Serialize CSS and then JS order in the downloads (not css/js/css/js)
Minify your JS using Jsmin
Avoid redirections, optimize pictures, and DNS queries

Breaking the 1.5 Second Barrier

Reduce I/O bottlenecks (RAID 10, cache and session on tmpfs RAM disks, don’t log everything)
MySQL – Split read & write to different DB:
- -> host it on a RAMDISK
- -> Host it on a Physical Disk
Some Advanced Tweaking
- Use Sysfence to autokill the bombing crons or php process
- Truncate (or archive) your log tables in Magento
- Check foreign includes response times and put them at the end of pages
- Use pagespeed / Yslow / Gtmetrix to analyse your site
- Integrate Lucene SOLR on large catalog database (> 100 000 products)
- Use latest version of Magento and activate all caches in the backend (if possible)
- more…
Analyze your traffic (throttle mailouts, same for all traffic generation – burst is your enemy)
Use cache the right way
- PHTML blocks should be cached it in order not to be generated each view
- To cache a block, you have to declare it explicitly
- Developers usually don’t use the Block caching mechanism at all (80%)
- Up to ten times faster loading when using cache properly!

These last two sessions were invaluable, and the information covered will assist us in scaling our Magento hosting options to grow with the needs of our customers. Keep an eye out for more posts on Magento optimisation. We’ll be doing our own R&D, both at the environment and application level. We’ll be providing extensive coverage of application optimisation, covering a full gamut of available techniques and providing benchmarks for each.