Some organisations continue to ignore the importance of proper enterprise scaling. Excuses I’ve heard include: “If it is a brand-name application it will scale fine” and “We’ll scale by throwing more hardware at it”.
Such ideas often stem from a heavy reliance on name-brand applications, and are not confined to government organisations, but are also found in industry server rooms. Although there is nothing wrong with using commercial applications, the error lies in believing commercial off-the-shelf (COTS) products will scale out of the box.
There is an erroneous assumption that using name-brand applications supplants the need for contextual scaling, which always requires technical expertise. Having the right technical competencies is essential to large scaling initiatives.
Perhaps few people understand this as well as John McMullen and the teams at Wellington’s Weta Productions and Weta Workshop. They have recently used scaled enterprise systems to help produce the award-winning animated children’s series, Jane and the Dragon.
To properly support such a major initiative, “forward planning is crucial”, says McMullen, Weta’s IT manager. His team wisely planned system upgrades based on feedback from several departments. This resulted in improvements in forecasting of technical requirements.
“As the complexity of Jane and the Dragon’s’ CG (computer generated) environments grew, we required more rendering power. However, we had reached the maximum number of processors we could use in our render wall,” says McMullen.
Making major scaling decisions, especially in the midst of deadlines, requires a technical team that is both sensible and skilled. The team at Weta was able to render the episodes on schedule using their existing resources.
Having the right combination of tools and technical experience also allows for the proper assessment of how applications will impact on scaling. Often, too fervent an emphasis is placed on the network and hardware layers, when really the application and the underlying platform play a significant role.
Beau Butler is an open-source design generalist working for Asterisk, in Auckland, and is experienced in scaling. He says that “by far the biggest aspect is the proper design of the application itself. Trying to retrofit scalability, cluster or grid models to an existing application is almost always doomed to failure – or at least [is] very inefficient.”
The proper design and tuning of applications requires technical proficiency that is often not available in-house. It is also not inherent in many COTS products. In most cases, it requires special skill and proficiency to understand how to tweak an application for maximum performance under peak-loads.
Asterisk engineers, who deploy their own Linux server derivative, are well aware that tuning an application is highly dependent on the proper tuning of the underlying operating system.
If these fundamentals are not considered, the result will be increasingly longer outages and, ultimately, far higher scaling costs. A full system re-architecture may be necessary.
Proper scaling depends not only on forgoing the myths, but also on realising that technical expertise is crucial to enterprise architectures. Relying on name-brand applications is reasonable only when combined with the right technical skills to ensure proper scaling.
Mark Rais is a Wellington-based technical writer who previously served as a senior manager at Netscape and AOL.