MemryX Is A New AI Company We Actually Need

Cloud MemryX Is A New AI Company We Actually Need Patrick Moorhead Senior Contributor Opinions expressed by Forbes Contributors are their own. I write about disruptive companies, technologies and usage models. New! Follow this author to stay notified about their latest stories.

Got it! Aug 11, 2022, 11:34am EDT | New! Click on the conversation bubble to join the conversation Got it! Share to Facebook Share to Twitter Share to Linkedin MemryX MemryX Artificial Intelligence (AI) is all around us, in the cars we drive to the smartphones we use, powering our most favored or despised apps. Consequently, one of the major growth drivers in the semiconductor industry is AI. It’s everywhere and revolutionizing the way humans interact with the modern world, from the datacenter to the device edge and everything in-between.

Running AI on the edge has become commonplace as these interactions need to happen quickly with low latency while minimizing the data transport expense. MemryX , a new startup with a focus on AI processing for edge devices, is creating new technologies to address this emerging market. But do we really need another AI chip company? I think we’re still in the second or third inning of this game and MemryX has the potential to disrupt the device edge semiconductor market with a combination of its unique, memory-centric AI architecture and its easy-to-use and scalable software/hardware solution.

The company’s leadership is both experienced and practical, knowing what it takes to ramp high volume semiconductors. If you’re not first to market, you must be very different and that’s where I see MemryX fitting in. Let’s take a closer look at MemryX, its direction, and how it might be different from the pack.

Company basics MemryX was co-founded in 2019 by Dr Wei Lu, MemryX CTO, to develop a leading AI accelerator for Edge devices. Dr. Lu has been an Electrical Engineering and Computer Science professor at the University of Michigan since 2005.

His expertise in memory systems and neuromorphic computing is internationally recognized . Dr Lu and his team spent nearly three years developing and proving MemryX’s unique approach. In December 2021, MemryX hired Keith Kressin as President and CEO to commercialize MemryX’s technology.

Keith has over 25 years of experience in semiconductor leadership and was previously a Senior Vice President and General Manager at Qualcomm. He helped guide technology and products at Qualcomm during the key growth phases of 3G/4G/5G smartphones, AR/VR, and AI initiatives. He also spent nearly a decade at Intel.

I know Keith well and I think he’s the right kind of guy to scale a company. Keith’s reputation enabled him to hire a strong leadership team in Engineering and Sales/Marketing bringing decades of executive experience from AMD, Intel, Micron, and Qualcomm. Experience matters.

MORE FOR YOU Western Digital’s Journey To Build Business Resiliency Through Cloud And ERP Transformation Amazon Climate Pledge: Two Years In And Going Strong Microsoft Takes First Steps To Finally Kill The Password The company has been frugal, having raised a very modest $11M in Seed and Series A, while successfully taping out multiple generations of chips to prove the technology works. It just recently announced customer sampling of their pre-production MX3 AI Accelerator hardware and software. I’m sure series B round isn’t far behind given the required financial levels it takes to be in the increasingly strategic semiconductor market.

MemryX is backed by several well-known investors including HarbourVest, eLabs, M Ventures, and Arm MemryX The Target and Flag Plant Unlike most AI startups that pretend to solve all things for all people, MemryX is focused on delivering an Edge AI inference (not training) solution for vision and sensing solutions. Its solution is an AI accelerator that complements, rather than replaces, a central processor for either new or legacy systems. So, it is trying to do one thing really well.

Overall, MemryX is planting the flag on delivering a high efficiency, low power solution, which benefits the end user. But oftentimes even more importantly, they are delivering an Edge AI solution that is the easiest to implement, benefiting the designer. The challenge The problem that MemryX is trying to address is a multi-variate challenge.

The growing diversity of smart devices makes it difficult to employ traditional computing architecture to run AI models at the Edge. Furthermore, AI systems are data centric, while classic processors are designed to execute instructions. In traditional computing, AI models are stored in DRAM resulting in bottlenecks, higher latencies, and higher power.

So many systems can run some AI, but not very efficiently. AI accelerators for edge devices are expensive to develop because of the difficulty and inconsistency of software implementation. It typically takes a tremendous engineering effort to create and update AI algorithms across a widening array of hardware solutions including ARM, x86, and now RISC-V ISAs.

And just as investments get made, ongoing changes to algorithms and operating systems continuously increases the need for further investment. As AI makes its way into nearly every chip and device, it becomes apparent why dedicated AI accelerators have limited market traction. A new approach to edge AI which addresses these issues could add tremendous value.

A solution with ‘One Click Optimization’ By moving toward a unique dataflow architecture with at-memory computing, much like how AI networks are naturally organized, MemryX should overcome the bottlenecks and complexity of software implementation and meet the needs of Edge AI at broader scale. MemryX’s Neural Processing Unit (NPU) has lower latency compared to other AI accelerators because of its inherent ability to store AI models on-chip rather than in DRAM. The proprietary dataflow architecture is also designed to minimize any movement of data within the chip, since moving data requires more energy than AI computations.

This enables their solution to maximize throughput while minimizing power usage. The architecture enables “one click performance optimization” delivering 50-70% utilization of the chip without the need to hand-tune any software. I’ll admit, I was a bit skeptical when MemryX briefed me on this.

It was almost too good to be true given hand-tuning AI models for weeks or months is common practice. I became a lot more convinced when the company shared with me a growing list of well over 100 models across 13 categories that run efficiently on its solution using all the same software. It also supports models across every popular AI framework.

Meanwhile, MemryX has just 30 employees. Impressed yet? The company’s latest chip, the MX3 is also scalable, so one can connect anywhere from 1 to 16 small chips, scaling the production version of the chip from 5 TFLOPs to 80 TFLOPs. Even more impressive is performance per watt given each chip averages only 1 Watt of power.

This means that MemryX with no software tuning, says it is over 5X more efficient than an NVIDIA AGX Xavier system that uses NVIDIA’s own proprietary software. I did not personally run the tests, but the test methodology passes my smell test. MemryX AI Accelerator(s) can be directly plugged into an existing USB or PCIe port.

MemryX Moving with momentum MemryX is only a little over three years old and has made great progress by my yardstick, completing its seed financing round in 2019 and just seven months later proving its proprietary architecture worked in silicon. This past year, MemryX produced its pre-production chip, which it announced this week was sampling to customers. The company expects volume production to start in 2023.

Since MemryX is producing such a focused product with the potential to scale in ways that have not been seen much of in the low power, AI edge market, I believe that if all goes according to plan, it could have a tremendous impact on the intelligent edge. MemryX says it’s working with alpha customers in several target markets including automotive, edge servers, metaverse, and security cameras. I think this gives MemryX a diverse portfolio of markets and multiple opportunities to experience significant growth.

Wrapping up While some might say the AI accelerator space is crowded, it’s apparent to me MemryX is different. It’s uniquely solving well known problems in edge AI. Its focus is on unique technology to meet real market needs is half the battle.

MemryX looks to be off to great start. I look forward to further disclosures and will share as appropriate. Moor Insights & Strategy, like all research and tech industry analyst firms, provides or has provided paid services to technology companies.

These services include research, analysis, advising, consulting, benchmarking, acquisition matchmaking, and speaking sponsorships. The company has had or currently has paid business relationships with 8×8, Accenture, A10 Networks, Advanced Micro Devices, Amazon, Amazon Web Services, Ambient Scientific, Anuta Networks, Applied Brain Research, Applied Micro, Apstra, Arm, Aruba Networks (now HPE), Atom Computing, AT&T, Aura, Automation Anywhere, AWS, A-10 Strategies, Bitfusion, Blaize, Box, Broadcom, , C3. AI, Calix, Campfire, Cisco Systems, Clear Software, Cloudera, Clumio, Cognitive Systems, CompuCom, Cradlepoint, CyberArk, Dell, Dell EMC, Dell Technologies, Diablo Technologies, Dialogue Group, Digital Optics, Dreamium Labs, D-Wave, Echelon, Ericsson, Extreme Networks, Five9, Flex, Foundries.

io, Foxconn, Frame (now VMware), Fujitsu, Gen Z Consortium, Glue Networks, GlobalFoundries, Revolve (now Google), Google Cloud, Graphcore, Groq, Hiregenics, Hotwire Global, HP Inc. , Hewlett Packard Enterprise, Honeywell, Huawei Technologies, IBM, Infinidat, Infosys, Inseego, IonQ, IonVR, Inseego, Infosys, Infiot, Intel, Interdigital, Jabil Circuit, Keysight, Konica Minolta, Lattice Semiconductor, Lenovo, Linux Foundation, Lightbits Labs, LogicMonitor, Luminar, MapBox, Marvell Technology, Mavenir, Marseille Inc, Mayfair Equity, Meraki (Cisco), Merck KGaA, Mesophere, Micron Technology, Microsoft, MiTEL, Mojo Networks, MongoDB, National Instruments, Neat, NetApp, Nightwatch, NOKIA (Alcatel-Lucent), Nortek, Novumind, NVIDIA, Nutanix, Nuvia (now Qualcomm), onsemi, ONUG, OpenStack Foundation, Oracle, Palo Alto Networks, Panasas, Peraso, Pexip, Pixelworks, Plume Design, PlusAI, Poly (formerly Plantronics), Portworx, Pure Storage, Qualcomm, Quantinuum, Rackspace, Rambus, Rayvolt E-Bikes, Red Hat, Renesas, Residio, Samsung Electronics, Samsung Semi, SAP, SAS, Scale Computing, Schneider Electric, SiFive, Silver Peak (now Aruba-HPE), SkyWorks, SONY Optical Storage, Splunk, Springpath (now Cisco), Spirent, Splunk, Sprint (now T-Mobile), Stratus Technologies, Symantec, Synaptics, Syniverse, Synopsys, Tanium, Telesign,TE Connectivity, TensTorrent, Tobii Technology, Teradata,T-Mobile, Treasure Data, Twitter, Unity Technologies, UiPath, Verizon Communications, VAST Data, Ventana Micro Systems, Vidyo, VMware, Wave Computing, Wellsmith, Xilinx, Zayo, Zebra, Zededa, Zendesk, Zoho, Zoom, and Zscaler. Moor Insights & Strategy founder, CEO, and Chief Analyst Patrick Moorhead is an investor in dMY Technology Group Inc.

VI, Dreamium Labs, Groq, Luminar Technologies, MemryX, and Movandi. Follow me on Twitter or LinkedIn . Check out my website or some of my other work here .

Patrick Moorhead Editorial Standards Print Reprints & Permissions.