Multi-core processors and the multi-processor render-walls of animation companies such as Weta Digital are two obvious faces of a move towards more parallelism in digital processing -- a trend that can work to New Zealand's benefit, local champion Nicolás Erdödy believes.
Tempting "big-science" prizes such as the Square Kilometre Array (SKA) radio telescope, for which Australia and New Zealand are bidding against other nations, will demand expertise in parallel processing, to deal with the massive flow of data to come from its observations. Other data-intensive disciplines at the forefront of science, such as genomics and climate modelling also demand parallel processing techniques in software.
The work of Erdödy and his colleagues at Oamaru-based Open Parallel has attracted the attention of Intel. James Reinders, marketing and business director for Intel's software developer products division, attended a Wellington meeting last week at which Open Parallel discussed the need for expertise in parallel processing.
The future of parallelism is the future of computing, Reinders says; around 2005, Intel stopped increasing clock speeds on its simple processors, facing problems of increasing power demand and heat dissipation. Instead, it used increased transistor density to put multiple cores on one chip, genuinely enabling the processor to do more than one thing at once, in contrast to the illusory concurrent activity of multitasking in the operating system.
Reinders is impressed with the work Open Parallel researchers and others have been doing to introduce parallel-processing commands to the PHP and Perl languages and thereby the WordPress blogging environment and other open-source products, using Intel's Threading Building Blocks (TBB) library. The projects implemented TBB commands into HipHop, a source-code transformer developed at Facebook. HipHop translates PHP into optimised C++, which it then executes in a particularly efficient way. Facebook open-sourced the HipHop code last year.
The Open Parallel team "made slight modifications to WordPress, to use a TBB-enabled HipHop they created -- and they had very impressive results," Reinders reports. "The bigger surprise was the huge drop in memory footprint." A task divided into several simultaneous sub-tasks can take up less memory than the same task as a single entity, he says.
Facebook, which developed HipHop to speed the rendering of its complex pages, is looking at the New Zealand parallelism work with interest, Reinders claims.
Open Parallel is organising a conference named Multicore World 2012, to be held from March 27-28 next year in Wellington. This represents an opportunity to get together vendors, researchers, software developers and the user community and encourage them to network and share their views of the parallel processing world.
"We aim to get 300 people to the conference," Erdödy says; many will come from overseas, but an important aim is to get New Zealand researchers and businesses talking to their peers and so build long-term collaborative capability.