Lab49’s Damien Morton has been doing work on “General Purpose Computations on Graphics Processing Units”.  His work is written up in a whitepaper and can be found here.

Matt also references some further chip core stuff here  and notes further GPU references.

Out of all the sessions the one that was the most interesting was one on Field Programmable Gate Arrays (FPGAs) - it also actually followed the outline described in the glossy brochure!

Although FPGAs can deliver up to 1000x faster performance than CPUs, the implementation may actually result in performance gains in the order of 40x or 200x since the developer needs to strike a balance between designing for pure performance and flexibility of functionality. Quoting the example that was given, the presenter had built a Monte Carlo simulation on a 15W FPGA chip that was 230x faster than a 3Ghz CPU, but in another solution, calculations were only 40x faster as they traded performance for flexibility. 

It would seem that most of the IBs are looking at Proof Of Concepts of FPGAs, and possibly implementing a “golden node” inside a regular grid.

One of the key messages was the relative difficulty in implementing FPGA solutions:

• Requires a higher ratio of engineering skills to modelling
• It is a human process rather than an automated one.
• Higher Development costs (there exists a 20-80 rule in that 80% of the work delivers only 20% incremental performance gain)

That said, the capital costs and operating expenses is considerably lower. E.g compare a 100,000 node CPU grid with a 100 node FPGA grid .. for same performance, although it will be harder to implement, it will be cheaper to run & operate

Although anyone trying to get into this field needs new engineering skills, equipment, etc, it seems that the only way that adoption is really going to happen is by convincing business users of the potential upside and getting them to sponsor the program. Functionally, it was mentioned that the best types of applications are either a) functionality that is relatively stable (for high throughout computation for well known models) or b) high-value functionality that merits high performance (scenario-based risk analysis for complex credit derivatives).

Building solutions on FPGAs does require a new engineering approach versus CPU-based solutions as you have to design for acceleration. Upfront design based on requirements is VERY important and highly human-based.

Other Related Stuff:

At Lab49, Damien Morton has done a bunch of work on GPUs . It will be interesting to see which way the banks go with non-CPU solutions.

Matt previously posted some info on FPGAs here

The second session of the conference was succinctly called “Enabling Financial Analytics for Competitive Advantage via High Performance, Scalable and Flexible Infrastructure” aka Big-Blue-Tooting-Its-Horn.

I guess the headline was the new form factor of their Blue Gene next-generation hybrid supercomputers which leverages their Cell Broadband Engine Architecture. Each rack in this multi-core (AMD/Intel) machine  has 1 Terabyte of memory and supports 2048 threads. It was described as a giant memory stick studded with chips and “pointers” where the 16,000 nodes can deliver 8 Terabytes of memory and perform 90 Gigachases/sec. IBM have built this machine with no moving parts (e.g. the fans are separate), helping to keep the temperature down. In terms of I/O integration the “memory stick” is studded with I/O chips to allow grid integration with both Datasynapse and Platform. Apparently, this can keep scaling, but starts to run into memory constraints.

Next up was the General Parallel File System (GPFS) which assists in the scaling of file servers and avoids the bottlenecks of NFS/SAN based file systems. The idea is that any node can read to/from any (shared) disk in the system. GPFS is not a client-server FS and stores metadata with the files and has no single metadata server. Performance-wise it allows access at the rate of 15GB/s for any single node and 100GB/s against any single file; supports 100s of nodes; and over 200Terabytes of storage.

IBM’s new BladeCenter was also profiled in terms of dealing with network latency, improved power output/heat density and supporting virtualization to control loads.

Finally, we got to the “Latency Stack” (not sure I want to buy a stack of latency!). This is another way of packaging all the new Websphere stuff that includes Websphere Extended Deployment (XD), Websphere Front Office for Financial Markets (mainly deals with streaming data apparently), and Websphere Realtime 1.0 (which has JVM extensions and allows control of the Garbage Collector and Ahead of Time (AOT) Compliation.

I attended the 2006 High Performance on Wall Street event yesterday and went to a bunch of the sessions. This event was considerably smaller and more focused than most events, and most of the usual vendors were exhibiting, including a lot of hardware vendors. Some of the major comments from various sessions are below.

General Panel Comments  - on a couple of the sessions, each panelist got to pitch their product or success story and give some insights. One key message that was reiterated several times was the need for better benchmarking and standards around grid. Some other specific comments included:

• Lehman’s Thanos Mitsolides claimed the toughest issues are around initialization and stateless execution. The problems not just about performance, but scalability, security, load balancing, etc are very important.
• BankOf America’s Andy Doddington - Being at the “top of the stack” there are needling issues at all layers in the stack. In the final session he re-iterated Thanos’ comments about HPC not just being about performance: management, visibility, ease of deployment and simplicity are very important too and stressed that vendors need to keep all of this in mind in their products. He was also complementary about Javaspaces as an easy way to manage data.
• Wombat said that there are lots of vendors coming up with what is essentially the same solutions. There needs to be standard measurements of products, as well as a better understanding as to where certain products apply to certain problem domains.
• Reuters’ issues are around the transformation of data and fan-out. They would like to see standard APIs from people like Intel, etc to support local transformation because they dont want to be hardware-dependent.

Visions for Future - in the opening session, some of the panelists were asked about what they see for the future:

• Lehman was concerned about the growth of clusters and what that will do to the scalability of the file system and backend cache infrsatructures.
• Gemstone said that they look forward to there being more stateful applications across more functions.
• Technology Business Development Corp said they want to see more benchmarks.
• Wombat  thinks that the issues going forward is not the technology itself but rather in technology management, so as volumes grow systems have to work with the datacenters/grids in place and so software efficiency will become more important.
• Platform thinks that the Quality Of Service and performance (fault tolerance, resiliency, measurement) are important especially as grid grow to 20,000 nodes in size. They also believe service-orientated infrastructures will be very important.
• Reuters expressed their needs for standards and benchmarking as well as monitoring of real-time latency. They also think that people need to focus on better infrastructure and how it is structured in the network.

More specific posts to follow.

I see Quocirca (and Oracle - they commissioned the article) have updated their Grid Index which shows “how initial pilots of Grid computing are now moving towards full implementations”.

According to the article, enterprise-wide grids are still rare and tend to favour more discrete cluster grids - in FS this is kind of consistent with business-aligned implementations of vendors such as Datasynapse and Platform. I also thought it was interesting that the US leads in adoption rates (we see the opposite effect in Financial Services where London is ahead of New York, for example).

There is mention of a tight correlation between localized SOA implementations and the use of grids, while broad-scale SOA adoption and grids are much looser correlated. This is not very surpising, as I have not seen many broad-scale SOAs in the first place, and even business-aligned SOAs in finance seem to have had limited success (as a real service-orientated architecture versus component-based architectures which is what most companies end up doing). At least the article’s conclusion about there being a low level of knowledge around SOAs confirms this. Not so sure that the reason is that companies do not want to combine too many new technbologies into one larger project, or if it is because businesses tend to be more stove-piped in general.

One thing I did not see referenced is the implementation of data grids, and distributed memory solutions in general which are definitely enjoying some growth in FS. Would also like to see a similiar study done that included virtualization and how it is actually being used.