Category Archives: Artificial Intelligence

Exascale Computing is at the Doorstep

The best current supercomputers are “petascale” machines.  This term refers to supercomputers capable of performing at least 1.0 petaflops [PFLOPS; 1015  floating-point operations per second (FLOPS)], and also refers to data storage systems capable of storing at least 1.0 petabyte (PB; 1015  bytes) of data.

In my 13 November 2018 post, I reported the latest TOP500 ranking of the world’s fastest supercomputers.  The new leaders were two US supercomputers: Summit and Sierra.

  • Summit:  The #1 ranked IBM Summit is installed at the Department of Energy (DOE) Oak Ridge National Laboratory (ORNL) in Tennessee.  It has a LINPACK Benchmark Rmax (maximal achieved performance) rating of 143.5 PFLOPS and an Rpeak (theoretical peak performance) rating of 200.8 PFLOPS.
  • Sierra:The #2 ranked IBM Sierra is installed at the DOE Lawrence Livermore National Laboratory (LLNL) in California. It has an Rmax rating of 94.64 PFLOPS and an Rpeak rating of 125.7 PFLOPS.

The next update of the TOP500 ranking will be in June 2019. Check out their website here: 

http:// https://www.top500.org

New exascale machines are only a year or two away

The next big step up in supercomputing power will be the arrival of “exascale” machines, which refers to supercomputers capable of performing at least 1.0 exaflops (EFLOPS; 1018  FLOPS), and also refers to data storage systems capable of storing at least 1.0 exabyte (EB, 1018  bytes) of data.  As you might suspect, there is intense international completion to be the first nation to operate an exascale supercomputer.  The main players are the US, China and Japan.

In the US, DOE awarded contracts to build two new exascale supercomputers: Aurora (announced in March 2019) and Frontier (announced in May 2019).

Aurora supercomputer concept drawing.
Source: DOE / Argonne National Laboratory

The Aurora supercomputer is being built at Argonne National Laboratory by the team of Intel (prime contractor) and Cray (subcontractor), under a contract valued at more than $500 million. 

The computer architecture is based on the Cray “Shasta” system and Intel’s Xeon Scalable processor, Xe compute architecture, Optane Datacenter Persistent Memory, and One API software. Those Cray and Intel technologies will be integrated into more than 200 Shasta cabinets, all connected by Cray’s Slingshot interconnect and associated software stack. 

Aurora is expected to come online by the end of 2021 and likely will be the first exascale supercomputer in the US.  It is being designed for sustained performance of one exaflops.  An Argonne spokesman stated, “This platform is designed to tackle the largest AI (artificial intelligence) training and inference problems that we know about.”

Frontier supercomputer concept drawing.
Source:  DOE / Oak Ridge National Laboratory

The Frontier supercomputer is being built by at ORNL by the team of Cray (prime contractor) and Advanced Micro Devices, Inc. (AMD, subcontractor), under a contract valued at about $600 million. 

The computer architecture is based on the Cray “Shasta” system and will consist of more than 100 Cray Shasta cabinets with high density “compute blades” that support a 4:1 GPU to CPU ratio using AMD EPYC processors (CPUs) and Radeon Instinct GPU accelerators purpose-built for the needs of exascale computing. Cray and AMD are co-designing and developing enhanced GPU programming tools.  

Frontier is expected to come online in 2022 after Aurora, but is expected to be more powerful, with a rating of 1.5 exaflops. Frontier will find applications in deep learning, machine learning and data analytics for applications ranging from manufacturing to human health.

Hewlett Packard Enterprise acquires Cray in May 2019

On 17 May 2019, Hewlett Packard Enterprise (HPE) announced that it has acquired Cray, Inc. for about $1.3 billion.  The following charts from the November 2018 TOP500 report gives some interesting insight into HPE’s rationale for acquiring Cray.  In the Vendor’s System Share chart, both HPE and Cray have a 9 – 9.6% share of the market based on the number of installed TOP500 systems.  In the Vendor’s Performance Share chart, the aggregate installed performance of Cray systems far exceeds the aggregate performance of a similar number of lower-end HPE systems (25.5% vs. 7.3%).  The Cray product line fits above the existing HPE product line, and the acquisition of Cray should enable HPE to compete directly with IBM in the supercomputer market.  HPE reported that it sees a growing market for exascale computing. The primary US customers are government laboratories.

TOP500 ranking of supercomputer vendors, Nov 2018
Source:  https://www.top500.org
 

Meanwhile in China:

On 19 May 2019, the South China Morning Post reported that China is making a multi-billion dollar investment to re-take the lead in supercomputer power.  In the near-term (possibly in 2019), the newest Shuguang supercomputers are expected to operate about 50% faster than the US Summit supercomputer. This should put the new Chinese super computers in the Rmax = 210 – 250 PFLOPS range. 

In addition, China is expected to have its own exascale supercomputer operating in 2020, a year ahead of the first US exascale machine, with most, if not all, of the hardware and software being developed in China.  This computer will be installed at the Center of the Chinese Academy of Sciences (CAS) in Beijing.

You’ll find a description of China’s three exascale prototypes installed in 2018 and a synopsis of what is known about the first exascale machine on the TOP500 website at the following link:

https://www.top500.org/news/china-spills-details-on-exascale-prototypes/

Where to next?

Why, zettascale, of course.  These will be supercomputers performing at least 1.0 zettaflops (ZFLOPS; 1021  FLOPS), while consuming about 100 megawatts (MW) of electrical power.

Check out the December 2018 article by Tiffany Trader, “Zettascale by 2035? China thinks so,” at the following link:

https://www.hpcwire.com/2018/12/06/zettascale-by-2035/

Deep Learning Has Gone Mainstream

The 28 September 2016 article by Roger Parloff, entitled, “Why Deep Learning is Suddenly Changing Your Life,” is well worth reading to get a general overview of the practical implications of this subset of artificial intelligence (AI) and machine learning. You’ll find this article on the Fortune website at the following link:

http://fortune.com/ai-artificial-intelligence-deep-machine-learning/?utm_source=howtogeek&utm_medium=email&utm_campaign=newsletter

Here, the relationship between AI, machine learning and deep learning are put in perspective as shown in the following table.

Def of deep learning  _ FortuneSource: Fortune

This article also includes a helpful timeline to illustrate the long history of technical development, from 1958 to today, that have led to the modern technology of deep learning.

Another overview article worth your time is by Robert D. Hof, entitled, “Deep Learning –

With massive amounts of computational power, machines can now recognize objects and translate speech in real time. Artificial intelligence is finally getting smart.” This article is in the MIT Technology Review, which you will find at the following link:

https://www.technologyreview.com/s/513696/deep-learning/

As noted in both articles, we’re seeing the benefits of deep learning technology in the remarkable improvements in image and speech recognition systems that are being incorporated into modern consumer devices and vehicles, and less visibly, in military systems. For example, see my 31 January 2016 post, “Rise of the Babel Fish,” for a look at two competing real-time machine translation systems: Google Translate and ImTranslator.

The rise of deep learning has depended on two key technologies:

Deep neural nets: These are layers of neural nets that progressively build up the complexity needed for real-time image and speech recognition. Robert D. Hoff explains: “The first layer learns primitive features, like an edge in an image or the tiniest unit of speech sound. It does this by finding combinations of digitized pixels or sound waves that occur more often than they should by chance. Once that layer accurately recognizes those features, they’re fed to the next layer, which trains itself to recognize more complex features, like a corner or a combination of speech sounds. The process is repeated in successive layers until the system can reliably recognize phonemes or objects…… Because the multiple layers of neurons allow for more precise training on the many variants of a sound, the system can recognize scraps of sound more reliably, especially in noisy environments….”

Big data: Roger Parloff reported: “Although the Internet was awash in it (data), most data—especially when it came to images—wasn’t labeled, and that’s what you needed to train neural nets. That’s where Fei-Fei Li, a Stanford AI professor, stepped in. ‘Our vision was that big data would change the way machine learning works,’ she explains in an interview. ‘Data drives learning.’

In 2007 she launched ImageNet, assembling a free database of more than 14 million labeled images. It went live in 2009, and the next year she set up an annual contest to incentivize and publish computer-vision breakthroughs.

In October 2012, when two of Hinton’s students won that competition, it became clear to all that deep learning had arrived.”

The combination of these technologies has resulted in very rapid improvements in image and speech recognition capabilities and performance and their employment in marketable products and services. Typically the latest capabilities and performance appear at the top of a market and then rapidly proliferate down into the lower price end of the market.

For example, Tesla cars include a camera system capable of identifying lane markings, obstructions, animals and much more, including reading signs, detecting traffic lights, and determining road composition. On a recent trip in Europe, I had a much more modest Ford Fusion with several of these image recognition and associated alerting capabilities. You can see a Wall Street Journal video on how Volvo is incorporating kangaroo detection and alerting into their latest models for the Australian market

https://ca.finance.yahoo.com/video/playlist/autos-on-screen/kangaroo-detection-help-cars-avoid-220203668.html?pt=tAD1SCT8P72012-08-09.html/?date20140124

I believe the first Teslas in Australia incorrectly identified kangaroos as dogs. Within days, the Australian Teslas were updated remotely with the capability to correctly identify kangaroos.

Regarding the future, Robert D. Hof noted: “Extending deep learning into applications beyond speech and image recognition will require more conceptual and software breakthroughs, not to mention many more advances in processing power. And we probably won’t see machines we all agree can think for themselves for years, perhaps decades—if ever. But for now, says Peter Lee, head of Microsoft Research USA, ‘deep learning has reignited some of the grand challenges in artificial intelligence.’”

Actually, I think there’s more to the story of what potentially is beyond the demonstrated capabilities of deep learning in the areas of speech and image recognition. If you’ve read Douglas Adams “The Hitchhiker’s Guide to the Galaxy,” you already have had a glimpse of that future, in which the great computer, Deep Thought, was asked for “the answer to the ultimate question of life, the universe and everything.”  Surely, this would be the ultimate test of deep learning.

Deep ThoughtAsking the ultimate question to the great computer Deep Thought. Source: BBC / The Hitchhiker’s Guide to the Galaxy

In case you’ve forgotten the answer, either of the following two videos will refresh your memory.

From the original 1981 BBC TV serial (12:24 min):

https://www.youtube.com/watch?v=cjEdxO91RWQ

From the 2005 movie (2:42 min):

https://www.youtube.com/watch?v=aboZctrHfK8