Artificial Intelligence in Hardware

After a decade at Intel, I learned something that will blow your mind about the semiconductor industry. The $600B chip market just changed forever. Here's why: → Generic chips are hitting a wall → AI workloads need custom silicon → One-size-fits-all is dead. But Broadcom + OpenAI just revealed the solution: CUSTOM AI CHIPS. • Tesla's FSD chip: 21x faster than GPUs • Google's TPUs: 80% cost reduction • Apple's M-series: 40% better efficiency • Amazon's Graviton: 20% price improvement Instead of forcing AI into generic hardware... what if we built hardware specifically for AI? The benefits are insane: - 10x performance improvements - 50% power reduction - Custom architectures for specific models - Direct chip-to-algorithm optimization - Massive cost savings at scale This is about RETHINKING THE ENTIRE STACK. From my manufacturing AI work, I've seen how custom silicon transforms production lines. Now we're seeing the same revolution in AI infrastructure. Sometimes the best solutions hide in plain sight 🌟 #AI #Semiconductors #Innovation #Manufacturing #TechTrends #DigiFabAI

AI Designs Computer Chips Beyond Human Understanding—A Breakthrough or a Problem? Key Points: • A neural network has designed wireless chips that outperform human-made versions. • The AI works in reverse, analyzing desired chip properties before designing backward. • Unlike AI hype, this research is peer-reviewed, open-access, and published in a reputable journal. • The concern: engineers may not fully understand AI-generated chip designs, raising issues of transparency, reliability, and security. Why It Matters Modern life depends on computer chips, and the race to improve efficiency, speed, and power consumption is relentless. AI can now design superior chips faster than human engineers, challenging traditional methods of hardware design. However, if humans don’t fully comprehend these AI-created architectures, debugging, optimizing, and ensuring security could become major challenges. What to Know • The convolutional neural network (CNN) used in this process learns chip design from scratch, creating architectures optimized beyond human intuition. • Kaushik Sengupta, an IEEE Fellow and electrical engineer at Princeton, led this breakthrough. • The AI-designed chips outperform traditional versions in wireless communication, improving signal efficiency and energy consumption. • However, the AI’s approach is a black box, meaning engineers can’t fully explain why the design works so well. Insights & Implications This advancement pushes the boundaries of AI in engineering, but also raises concerns. If engineers cannot fully understand AI-generated chip designs, troubleshooting, security audits, and long-term reliability could become serious risks. Additionally, AI-designed chips could contain vulnerabilities that go unnoticed, making them potential targets for cyber threats. While this technology has game-changing potential, experts must balance innovation with accountability, ensuring that AI remains an assistive tool rather than an opaque, uncontrollable architect of critical infrastructure.

I just published my latest piece exploring the architectural bottlenecks of today’s dominant AI models—and why the future lies in brain-inspired computing. Despite the transformative impact of transformers and deep neural networks, we’re hitting hard limits: 1) Unsustainable compute demands 2) Poor long-range sequence handling 3) Diminishing returns from scaling 4) Opaque decision-making To move beyond these constraints, I argue for a paradigm shift—toward biologically plausible systems, such as Spiking Neural Networks and neuromorphic hardware. These models offer energy efficiency, adaptability, and a path toward truly general intelligence. If you're working at the intersection of AI, neuroscience, or systems design, I’d love your thoughts on how we bridge the gap between algorithmic innovation and hardware realities. 🔗 Read the full article: https://lnkd.in/gmx7b7z9 #AI #AGI #NeuromorphicComputing #BrainInspiredAI #Transformers #TechEthics #InnovationStrategy

AI: If Data Is the Moat, then is Hardware the Shovel? For years, many VCs and angels have been cautious on hardware: capital intensive, slow to iterate and often hard to scale. But recently, my investment philosophy has started to shift. Why? AI has led to extraordinary software innovation, but also a lot of duplication. Many startups who’ve built thin wrappers on top of foundation models may have little defensibility beyond UX and distribution. Instead, some compelling AI companies are solving real-world problems by collecting high-quality, often previously inaccessible, data. That might mean sensing something new inside the body or the natural world - but it starts with owning the hardware/sensor. In some sectors, such as medical devices, this also comes with regulatory complexity, but this also creates barriers to entry and adds to long-term defensibility. Real-world data, especially when collected at the edge, can make AI practically useful. In many applications, on-device inference is essential for latency, privacy, or power reasons. The closer you are to the data source, the richer the signal and the more context-aware your models can become. What’s also helped change my thinking is that hardware development cycles have compressed significantly. With rapid prototyping tools and off-the-shelf components, what once took years (and £££) can now be built and tested in months. Yes, tt’s still harder than software, but the gap is narrowing. Finally, this proprietary data collection doesn’t just improve performance, it can create a sustainable competitive advantage; if your system captures signals no one else can access, you then control both the input and the insight and this can be hard to displace…. 

Nine months ago, I published my paper on designing an AI hardware accelerator from scratch—a challenging yet rewarding journey. Unlike the common trends of binary computing, analog approaches, or neuromorphic designs, this project pushed the boundaries of digital logic by rethinking information theory. A deep integration of hardware-software co-optimization demanded versatility and proficiency in both high-level and low-level programming, and the creation of custom design automation to manage the vast complexity of CNN parameters. Recently, I came across one of my screenshots on MNIST CNN classification. It felt unreal at first but realizing that the results were on actual hardware turned that feeling into pride. It’s a glimpse into the potential of FPGA in future AI hardware by achieving CNN classification in nanoseconds. No AIE, no DSP, no BRAM, just pure LUT horsepower. https://lnkd.in/g8CrUJqX #AI #FPGA #Innovation

AI is advancing at a pace that makes Moore's Law look downright sluggish. In just five years, the compute demands of cutting-edge AI models have grown by a staggering 40,000x. So how can we build hardware that keeps pace? That's the central question I explore with Sean Lie, co-founder and CTO of Cerebras Systems. Sean takes us through Cerebras' journey, from recognizing AI's unique computational needs to their bold decision to build wafer-scale processors. It's a story of genuine deep tech innovation that challenges long-held beliefs about the limits of chip manufacturing. But Cerebras isn't just supersizing chips. Sean explains how the team is reimagining the entire AI stack—from chip design to server architecture, power management, and even the algorithms themselves. Their holistic approach underscores that true breakthroughs stem from rethinking entire systems, not just tweaking individual components. Whether you're knee-deep in AI research or a fellow hardware founder, our conversation offers a glimpse into the future of computing and what it takes to tackle today's hardest technical problems. Our full conversation here: https://lnkd.in/giGSvCXC

Deepseek has disrupted AI across many dimensions. One of them is doing more work during inference at the expense of training. Increasing inference time compute seems like an intuitive “Pay per View” tradeoff and other models are rapidly adopting it. Inference workloads already use twice as much of computing as training. The balance is likely to tilt more that way. Would this make a measurable impact on data center hardware mix? Inference workloads are distributed across a greater diversity of hardware platforms such as CPU, GPU, FPGA, ASIC, each of which exhibit distinct characteristics which can help improve LLM inference performance. CPUs excel in programmability, GPUs have massive parallel capabilities and memory bandwidth, FPGAs and ASICs are often designed for specific applications with the customized architecture offering higher computational efficiency and better energy efficiency. Different hardware platforms may also be combined to generate optimum tradeoffs between performance, accuracy, power, and cost. Optimizations for inference can also offer tradeoffs. These include quantization of weights and values (integer vs. float, bits of precision), selection of different evaluation operators (linear vs. non-linear), short-cuts like skipping layers in the model etc. There are four times as many CPUs shipped annually into data centers than GPUs. The installed base of CPUs is also much larger. Undoubtedly efficient use of capital favors using CPUs as much as prudent for inference. AMD, the current darling of server CPUs says this: “A powerful foundation for AI workflows, 5th Gen AMD EPYC processors are the ideal CPU-based AI platform to run inference across a variety of models and use cases. (They) deliver the flexibility to support requirements ranging from real time inference to batch or offline inference.” Investors seem bullish on CPUs even when (unlike x86 server chips) current shipments and installed base are miniscule. Softbank recently announced the acquisition of Ampere - one of the only merchant vendors of ARM based CPU server chips. AheadComputing a startup by Intel veterans secured healthy seed funding to rapidly develop and commercialize breakthrough RISC-V microprocessor architecture for computing demands across AI, cloud, and edge devices. Industry veterans like Jim Keller and David Ditzel are heading companies working on chips for AI that are substantially based on CPU and CPU performance. The Deepseek disruption is remarkable for arriving so early in the AI cycle. It has focused minds on reducing the end user’s cost for profitably using and deploying AI widely. This mission will have major implications for AI hardware as well. The CPU emerged as the Swiss army knife of computing after the first microprocessor on a chip was designed instead of an ASIC for a business calculator 50 years ago. Its Swiss army knife value might continue to help it thrive through the AI revolution. 

Artificial Intelligence can look intimidating - “black‑box” algorithms, pricey hardware, teams of PhDs. Yet remarkable results are possible with modest gear and a bit of curiosity. Take 17 year old Ben Choi. Instead of implanting electrodes in the brain (a procedure that can cost hundreds of thousands of dollars), he placed postage stamp sized sensors on the skin of the forearm. These sensors pick up the tiny electrical pulses that our brains send to muscles signals so small they’re measured in microvolts. Here’s where AI enters the story: - Signal capture: The surface sensors record raw voltage changes every few milliseconds. - Pattern learning: A lightweight machine learning model (think of a mini neural network running on a laptop) studies those voltage patterns and learns to match them with the user’s intended hand motions - open, close, rotate, and so on. - Robotic action: A 3D printed arm receives the AI’s instructions and moves accordingly, almost in real time. Because everything runs on off the shelf parts - an Arduino microcontroller, free Python libraries, and affordable hobby grade motors - Ben kept the parts bill under US$300. That price point matters: sophisticated prosthetics and assistive robots typically run well into five or six figures, placing them out of reach for many people who need them most. Projects like this shows that: - Open source tools lower barriers: Frameworks such as TensorFlow, PyTorch, and Scikit‑learn put advanced algorithms a few commands away. - Community knowledge compounds: Tutorials, discussion boards, and hobbyist forums mean you rarely start from scratch. Yes, AI raises legitimate concerns - bias, misuse, security. But it also unlocks practical solutions that improve lives: smarter medical devices, safer vehicles, more intuitive home tech. Have you seen other low cost, high impact AI projects? #innovation #technology #future #management #startups

AI is eating the world... And the power grid. As AI models grow, they are using more and more power. This is creating a major energy crisis in the industry. But what if AI could run on light? Not electricity, but light. Researchers at the University of Florida have built a new computer chip. It uses laser light for AI tasks. This chip is up to 100 times more energy-efficient. This is not just a lab experiment. The chip was tested. It classified handwritten digits with 98% accuracy. This is the same performance as standard chips. This is a big step forward. One of the researchers said it's a "𝘭𝘦𝘢𝘱 𝘧𝘰𝘳𝘸𝘢𝘳𝘥 𝘧𝘰𝘳 𝘧𝘶𝘵𝘶𝘳𝘦 𝘈𝘐 𝘴𝘺𝘴𝘵𝘦𝘮𝘴." The next race in AI won't be about speed. It will be about building more sustainable, efficient tech. Will the next big breakthrough in AI be hardware, not software?