The Cloud versus The Grid
Can we design renewable energy the way we design digital services? The Cloud and The Grid are not just metaphors, but the largest machines that humans have designed and made.
The cloud can be viewed as an emergent, bottom-up, decentralized system powered primarily by software. In contrast, the electric grid is a deliberate, planned, centralized structure that operates at the level of territories or nation-states, mainly defined by hardware.
Lunar Energy's Matt Jones explains why working on renewable energy requires us to create new ways of designing.
This talk takes us through projects from the last 25 years, starting with the shift to the cloud (aka "someone else's computer") and to where thinking about the grid could point us next.
We're super grateful to be here at AHO. And a lot of that is thanks to Mosse, and thanks to Einar and many of the other wonderful people here. And it's really a unique thing to have.
This wouldn't happen without lots of volunteers. This is the gang. We also have Jake and there's Matteo. If you are interested in joining us as a volunteer, Lene, who will now raise her hand, talk to her afterwards. None of this happens on its own.
Being able to come to wonderful venues like this and bring all this equipment and stream to everyone at home does not happen for free. So we're very grateful for the continued support of our sponsors who have been with us for years and years. And the funny thing is that designers don't like to talk about money.
So that means that a lot of grassroots design organizations never really got off the ground because they didn't talk enough about money. So thank you sponsors for being able to put up with our money talk, because it is actually a requirement to be able to do anything over time. You see 20 years ago we did have cool meetups, but there would usually just be one meetup and then the organization would collapse because it was just too much work to do it for free.
So all the wonderful corporate people looking now to figure how can I sponsor, you go and talk to me later. I'm really looking forward to this.
Thank you for coming and thank you to Mosse for connecting us all and making it happen.
Matt Jones My name is Matt Jones. I'm the head of design at a startup called Lunar Energy. We're focused on transitioning homes to 100% clean energy, clean electric power. I'm going to talk more about that later.
What I'm going to try and talk about tonight are the two machines that I'm trying to work with in that role, and how I've worked with them on and off for the last 30 years or so, which is kind of depressing now I say it out loud. I'm going to try and talk a little bit about the nature of these machines as well.
First of all, as the title of the talk suggests, the cloud and the grid. So the cloud or the internet, thinking of it more as, you know, you could also replace sort of accidental with emergent. You could replace kind of deliberate with planned. You know, the electric grid is a sort of huge planned machine, one that sort of operates more, I've sort of said of the size of planets, but it more properly operates at the size of territories or nation states.
It's not nearly as distributed across the world as the internet, and it's much older, you know, it's by some measures perhaps, you know, a hundred years old or more. So, you know, you can also replace accidental with emergent, planned, bottom-up, top-down, software, hardware, ah, my terrible fingers, decentralised, centralised, And so that's a little bit of what we're going to talk about tonight. Oh, oh, sorry everybody. And finally, you know, one is perhaps defined by software and the other defined by hardware. But we'll see how that is changing also a little later.
Let me see if I can do this with a light touch. Oh, no I can't. Who has heard of this, these two phrases? A little show of hands. Not many people. Okay. So, these come from an essay which is about the same age, if you like, as the World Wide Web. An essay by Brian Eno called "The Big Here and the Long Now" in which he sort of describes two concepts. One which is the long now, a phrase designed to conjure a more long-term framing of thought, I suppose. But the other phrase that accompanied it, which gets less coverage, I suppose, in his essay, is "The Big Here". And "The Big Here" is less familiar, but it's more exciting to me in some ways. And it refers to the deep intertwining of the local and the global, of the intertwined scales that we live in. the local, the personal, the political, the technical, the ecological.
And Eno's sparring partner in the Long Now Foundation and the birth of the clock of the Long Now and all of these sorts of ideas was Kevin Kelly, who's also written a lot about a kind of ecological conception of the technology that surrounds this, his notion of the technium. But Kevin Kelly also made this, which is one of my favorite things. It's called the Big Here Quiz. Are you really here now? You live in the big here. Wherever you live, your tiny spot is deeply intertwined with a larger place, embedded, fractal-like, into a whole system called a watershed, which itself is integrated with other watersheds into a tightly interdependent biome.
At the ultimate level, your home is a cell in an organism called a planet. All of these levels interconnect. What do you know about the dynamics of this larger system around you? Most of us are ignorant of this matrix predating the movie. But it's the biggest interactive game there is. Hacking it is both fun and vital. And so, again, about probably 12, 13, 14 years ago, Myself and a few other folks tried to do this with people in a city, in Brussels, as I recall, and asked them about what they knew about where they were. And if you go to look this up, the Big Here Quiz, it's a set of questions, starting out from where you are and almost pulling on the wool, on the thread of this, to go out into the larger ecologies that you're embedded in. It's really, really interesting. It's really hard, but it's really worthwhile doing. It's very, very inspiring to me, at least.
Alongside this, you know, the cloud, if you like, is also kind of something where we can sort of think about the big here of it. My friend Andrew Bloom wrote this wonderful book. Some of you, I know, have read it, called Tubes. And it starts off with Andrew, who's a journalist, a writer, in his apartment in Brooklyn, calling his internet company and saying, you know, "My internet is down." And he's like, "It's been down all morning. What are you going to do about it?" And the person on the other end of the phone says, "Hey, can you look out the window for me?" "Um, sure." "Do you see squirrels?" And he's like, "Yeah." He's like, "Can you see if they're, like, chewing on anything?" It's like, yeah, they're chewing. "Oh, yeah, they're chewing on that cable that's coming." "Yeah, yeah, this happens a lot this time of year". And he was like, wow, this is thing, which I've always assumed is like this, you know, kind of amazing, magical sort of property, and it can get, you know, eaten by squirrels. And so he literally starts pulling on that thread, that wire that the squirrels were chewing on. And he ends up going around the world, following the wire, and finding out what the internet is made of. And it turns out it's made of, you know, and most people now have seen many, many documentaries and things about this, but at the time, I think it was kind of novel way to look at it. But also, you know, a lot of this is kind of, in his book is about how it's made of people, and how it's made of agreements between people. Really, really wonderful book, but it's a sort of almost like a travel book. It's like travel journalism about the internet.
But what he's looking at is, you know, what kind of more, you know, if we're sort of stepping into, you know, writing our dissertations kind of land, what he's talking about is this idea of a Hyperobject, is a coinage by the philosopher Timothy Morton of something that's so massive and so distributed in the world that we can't really touch it or see it, but it touches and sees us. And what Andrew was doing, and what Kevin Kelly was doing with the Watershed Quiz, is it's very sort of grounded and embodied way of trying to understand these hyperobjects, these things that are too big to understand or see directly, but touch our lives every day, whether it's the global financial system, or the carbon cycle, or the internet, or a watershed.
And the other thing which has always been part of my work and work of studios that I've been part of has been this idea of trying to surface those. And one thing that we sort of talked about a lot back in Berg days was this idea, which we sort of got from John Thackara, of designers making macroscopes. The original coinage was that of Howard Odum, an ecologist in the 1970s.
But we invented microscopes to see very small things. We invented telescopes to see things which are very far away, but we are kind of lacking in inventing macroscopes. And macroscopes are things which we can use to see and understand these hyperobjects, these things which are very big, but right on top of us and in our lives every day. And of course, this is work that pioneering designers have done forever. We can't talk about understanding scale and understanding kind of making macroscopes referring back to this thing.
Who's seen this before? I would be shocked, shocked, if there weren't more hands raised than that. Charles and Ray [Eames] is amazing, "Powers of 10," which I think I saw when I was a small child and I think is to blame for much in my life. Perhaps the original work of designers working in macroscope making. Anyway, so this is a little bit of, you know, talking about work, but also talking about, I guess, my work. So this is my sort of like long now.
I started in an architecture school in Wales in the 1990s. And as I said, sort of about 30 years ago, the World Wide Web got invented and I got distracted from trying to be an architect, and wandered into working in technology and working particularly with internet technologies. So the other sort of way to, I've sort of tried to macroscope myself, if you like, is by stepping out of that sort of human scale and looking at kind of the machines I've been working with, I suppose. So this is kind of like my personal big here and long known way. I mean, I'm sure everybody here is familiar with Moore's Law. Anybody not familiar with Moore's Law? Okay, so, you know, it's roughly, you know, the fact that things get more cheaper, you know, cheaper and more powerful in terms of computer chips, transistor technology, those kinds of things, processing power every 18 months or so. And so I thought about this and sort of my lifespan.
So I was born in 1972. I was born in 1972. I just said that. Thank you, slide. So, you know, 1980s, I get my first computer, you know, I'm watching Star Wars, I'm being a nerd. But, you know, we're still at kind of like 50,000 transistors. We're still on, you know, my first computer was a VIC-20, if anybody remembers those. I went to college, I went to an architecture school, 1990 to 1995. And this was kind of the time that Pentiums came out. So that's what that number was referring to. And we were using AutoCAD on Pentium computers that had a button on them which said "Turbo". And anybody who's old enough to remember computers had buttons on them that said Turbo. And I'm sure that is the same as those buttons in elevators that don't actually do anything. So like, you know, web 1.0 happens, I get distracted. Web 2.0, coincident with a lot of kind of work at Berg, kind of happens there. And you're already starting to see, you know, this kind of incredible, incredible log scale happening.
And so, you know, effectively, you know, the stuff we work with, I was talking to Mosse earlier about this, you know, the rug, the carpet kind of gets pulled under us every 18 months. There's no sort of sense of history in the materials of what we do, because it changes so fast. The stuff we work with now is 50,000 times more capable and complicated than when I began getting paid to do this stuff. And it's a million times more complicated than in the 1980s. And that's kind of a very unusual thing for a sort of design material, a design practice to be involved with. You couldn't really say the same particularly of architecture or furniture design or sort of physical product design. There just isn't that kind of, you know, rapidity of fundamental change in the architecture of the materials that you're working with and the capabilities that it represents.
And so, and this is kind of, again, coming back to the kind of long now mafia who's familiar with this, This is Stuart Brand's pace layers diagram, almost compulsory to have it in an interaction design talk. But this is kind of interesting to me in a way because I sort of see this as layers of semiconductor, not kind of things that are shifting kind of over each other, but actually talking to each other. And I think one of the things that's really interesting about these two huge machines is they emerge from different layers, but they start to short circuit and influence all of these different layers. So, coming back to this idea of making macroscopes, all we can really see of systems are the things that we can experience. We can be told about them, we can have them explained to us, but ultimately we experience them.
And much of the complexity of the modern world and the systems within it, these things which are on top of us, affecting us, these hyperobjects, is not evident or even deliberately made invisible, right? And these, particularly these machines we're talking about, these two planetary scale machines, are deeply in need of this kind of macroscopic interrogation and critique, I believe. And of course, you know, even how I'm showing this, right, is metaphor, right? It's not, they're not really icebergs. And photons aren't really wiggly lines. Only, well, sometimes they are, right? And sometimes they're not.
But metaphors are powerful, and it's what we work with most of all in terms of interactions, for sure, is sort of poking the sort of metaphors in somebody's brain with metaphors that we design. So in this talk, I'm going to look at all of the work that we did at Berg, and also some of the work that I did at Google, and some of the work I'm doing now through these lenses, and also then come to what I'm doing right now, and how it connects to that other giant machine, the grid, electricity, and the energy transition.
So first of all, the cloud, you know, this is the king of metaphors, in a way, right? I'm sure most of you know that the cloud originates from what network engineers would draw on the whiteboard or on their documents when it wasn't their problem. Literally, it was like, "Okay, this is all our stuff, and then this is the cloud." So anybody, somebody talks to you about cloud computing or anything like that, obviously, I think a lot of us have seen t-shirts and mugs and stickers like this. It's absolutely true. It's somebody else's computer. But it's also the origin of the metaphor was literally that it was not our problem. And that's how we treat it, right? That's why we love it. Because it's like all of that stuff, not our problem anymore.
And so I'm not going to sort of labor the definition of the role of metaphors in design. design. There's lots of brilliant writing about that, and particularly in the sort of realm of what I was until recently looking at, the kind of design for systems involving AI.
There's loads, this recent paper by Dave Murray-Rust and Dan Lockton, I'm sure some of you know, and Iohanna Nicenboim is a really, really great read. I recommend that one. But what I want to also look at is this sort of relationship between architecture, materiality, and metaphor in both of these sorts of things. 'Cause I think it's always really worth, as I was sort of trying to point out that kind of stupid story about the cloud, you know, it's always worth questioning the default metaphor and where it came from and looking at its sort of, its social history in a way and kind of who is sponsoring or creating or hyping up that metaphor. And what does that mean for the kind of default architectures of what we build, the sort of technium around us? Because I think if you don't challenge those defaults, you both kind of miss some of the dynamics of power around these things. But also, just as designers, I think you miss opportunities. You just kind of don't get to see what the default path of the world might be obscuring or not letting you play with.
And so, first of all, gonna talk a little bit about my time at Berg. So that was about 2008 to 2013, I guess. Yeah, and Berg as a studio lived from 2009 to 2014. So the studio formed out of a partnership between Matt Webb, who's been recently teaching here in Jack Schultz, Another colleague of ours, Tom Armitage, joined, and then I joined shortly after that. And we grew to about 15 folks at about our largest, but it was always about 8 or 9, which was a nice size. Still really proud of the stuff that we worked on. And this slide comes from talks that we gave back in Berg days, and it was a kind of central tenet of the work. We believed that the choices as designers that you make can start to help people unpack the systems around them. And most of the design orthodoxy at the time, and still to this day, was around this sort of like drive to seamlessness and sort of ideas of, you know, the best UI is no UI and invisible computing and all of these sorts of things. And because we were also just, you know, liked an argument mainly, we were like, "No, that sounds stupid." You know, it's kind of like, why does that equate with, you know, enjoyment and beauty and ease of use? Why can't it be evident and beautiful and delightful and something pleasurable?
And so we really, like, you know, digging into the history of ambient computing and sort of environmental computing, looking at the work of Mark Weiser at Xerox PARC in the 1980s, where he talked about not making things seamless, but making beautiful seams. And that made perfect sense to us and became a sort of mantra for us, because if you look at anything that we prize in terms of material world, you know, pianos, architecture, cars, etc. Often, you know, we are looking for things which are really premium and very sort of prized, of the things where we really sense that craft in the seams. or we're able to see how something is put together and then build a metaphor on our heads, or a sort of folk model of how something is working from what the maker has, the choices the maker has made and made evident towards you.
And why couldn't we do that with technology? Why couldn't we make legible systems that were attractive and engaging? And another sort of mantra, central sort of tenet of this studio was like, "What just got cheap? What just got boring?" Another sort of quiz to the audience, who's familiar with the Gartner hype cycle? Some people. So this is Gartner, a technology analyst. Every year they publish a report and they look at sort of hyped terms, whether it's, you know, kind of like the metaverse or NFTs or things like that. Most recently, coming from a technology trigger to a sort of peak of hype. And then there's this wonderful thing called the trough of disillusionment, which is kind of when everybody sort of like gives up on a piece of technology and goes, "Oh, well, that didn't really work out, did it?"
But often because it got hyped up, people made millions of them, and it got cheap, and it got boring, and no one wants to touch it anymore. That's when it gets fun. So the idea of kind of being able to sort of, again, like, look at where, as we sort of said in big days, Moore's Law meets Main Street, what just got really, really cheap, commoditized, tough, resilient, something that you would hand to anybody without a second thought, rather than being a sort of prized piece of technology. And that sort of felt like the soup, the kind of base pairs of DNA of something that you could invent with, rather than looking at solely the sort of cutting edge of technology for inspiration when you're inventing things.
And another sort of thing that was, you know, hugely influential for me was a book called "The Nature of Technology" by this chap, Brian Arthur, who's an economist, a network economist, and he was curious why technology had created so much value in the 20th century and the early 21st century. And it's a book looking at technology through the forensic eye of an economist, and it's really, really interesting. He tries to establish how and why technology evolves and builds in value. And so he talks about assemblages of scientific and engineering phenomenon coming together, phenomena coming together to combine into new inventions. And so the give and take between sort of user needs and cultural needs and sort of these emergent phenomena were the things that sort of felt much more compelling and inspiring, again, to that kind of orthodoxy on one side of kind of like seamlessness and No UI, and on the other side, the kind of, you know, the kind of, the sort of mantra of IDEO, and it's the kind of like human-centered design thinking kind of stuff.
It felt like there was something more interesting that was more phenomenological at play. And I couldn't come here without mentioning this early piece of work of the studio done in tandem with Einar and Jørn and others at AHO, looking at kind of these phenomena and trying to chart them and trying to record them and trying to make them evident and legible, but trying to do that in a way that was, you know, beguiling and engaging. And so this is kind of a piece of work looking at RFID readers, the sorts that you get, you know, in touch interactions with credit cards and travel cards and those kinds of things, and poking an LED into the field of, the sort of reading field of this device, this kind of very blank black box to reveal this kind of shape of the immaterial, the sort of qualities of it. We really enjoyed examining things and sort of trying to unpack this kind of seamlessness into something that was, you could start to see the sort of beautiful seams of the technological world around us. And this sort of work was almost the precursor to the product invention. So this sort of gave us things to invent with in the way of Brian Arthur.
So the emphasis on sort of exploring phenomena, we kind of called material exploration. It was always the first phase of every project in the studio. We kind of led with it. We kind of privileged it in the way that our other contemporaries privileged, user research, sometimes to our detriment. We did barely no user research at all, sorry. I'm gonna get drummed out of the iXDA conference now. But this was what Vestudio was a sort of vehicle for, was this kind of material exploration that would then lead to sort of work. It's what powered it. And this is one of, this is a great example. This has got a little bit of sound, so hopefully you can hear it. This is work by Timo and Jack, Campbell Orr and Matt Brown.
"That's all from Newsnight tonight. Before we leave you, an advertising agency called Dentsu London and a film company called Berg were playing around with tablet PCs recently in search of ideas and came up with this. You take an iPad and move it along while the screen flashes up a series of 3D letters as seen in cross-section like this. And if you photograph what you're doing on one single long exposure shot, it looks like this. So they made a movie of it. And here's an extract. Good night."
No user research in the world is going to tell you to make this. But this is what comes from that kind of material exploration and sort of that phenomenological unpacking of technology. And then seeing how it meets with culture and meets with human needs, I think is how we saw the way forward. Another project we did which was very material exploration heavy, which was done for Google, was this idea of a project we called Lamps. It was the very early days of computer vision, commodified computer vision, it just starting to get cheap, if you like. It was sort of coincident with sort of things like Sony, the Sony iPlay webcam and the Microsoft Xbox. I've forgotten the name of it now. Connect, thank you. And so this work was about, okay, this is starting to happen, but what would it mean to sort of have computer vision in the world with you? Not through a screen, not through glasses. It was also coincident with Google releasing Google Glass, which we thought was terrible, but we couldn't tell them that, but we sort of did this and charged them for it.
But this was sort of like, almost like the alternate universe from Google Glass, where it's not individualized, it's not privatized behind an individual's pair of glasses, but it's in the world with people. And what would that feel like? And so a lot of this work was kind of like, again, like very, very sort of material exploration heavy, looking at sort of qualities of light and recognition in the world. What does it mean to have like light cursors in the world with you? What happens when you sort of see what the computer vision is recognizing or reading? What does that start to tell you about what the systems behave? Like what do they sort of privilege? How do they perform?
But because it's 2010, you have to do it this way. Which I still get a little bit of PTSD from and I wasn't even the person making this. Andy Huntington made this amazing rig, which now, like any of the work that you're seeing here, again, coming back to that kind of chart of Moore's Law, any one of you could do in an afternoon. It's kind of, you know, heartbreaking in some ways, but wonderful in other ways. Again, that kind of rug gets pulled from underneath all of the work. And lastly, I'll just quickly show this piece which was the sort of some of the final output Which was kind of like more sort of speculative design work about products that are brought to life by this kind of smart life and smart light in the world this combination of projection mapping and Computer vision and so these are dumb objects that are then animated by smart light So in a way, this is sort of really sort of making a point about where the where the computing is. It's in the cloud It's just being projected on these wooden blocks effectively. This is a this is a device for playing YouTube, which is nothing more than some blocks of wood and some Sort of registration marks for for the projector which and you can sort of see the queue up the sort of fiducial code 2D barcode kind of thing on the knob there. And one of the interesting things about this is because the computer has to see the control surface being turned by the user, it can't be on the control surface. It has to have a little pulley behind it so that it's not occluded by the hand. And so there's lots of really nice little thoughts and details in here, which are sort of interestingly critical when you start to unpack them.
This is work which we haven't really shown before, which I dug out, which is, but we never went public. This was stuff that we did for Intel around sort of connected cars and sort of thinking about like the car as a sort of media play head in the city, sort of interpreting the city connected to other cars. We did some sort of like video prototyping work around this. And it's kind of interesting to kind of see where this kind of went. You know, a lot of the stuff as well we were doing, again, because we just liked an argument, we saw more and more touchscreens coming into cars. We were like, "That's a terrible idea. "Don't put touchscreens in cars. "You gotta drive, you'll hit every," you know. It's taken about 15 years, but it seems like people are catching up with this, that it might be a terrible idea.
So we were looking at the car as a sort of more ambient computing control surface in some ways. And then we sort of also, you know, almost like recycled some of our obsessions into turning sort of this smart light idea into something about kind of involving the car owner in the health of the car. You know, modern cars are very much black boxes that you can't understand. You have to take them to official dealerships and those kinds of things. This was the idea of a sort of computer vision powered inspection lamp that would kind of like help you understand what your car needed and did.
But this was kind of like part of the talk that I gave back then, back in sort of like 2011 or something. And this is kind of, I dug this out 'cause I think it's kind of interesting.The idea that there is no separate, this was sort of like our little internal manifesto. There's no separate digital world. Software acts on the world around us. software acts in the world with us. It is a material to be designed with make it evident, make it beautiful.
Um, and then, um, we stopped. Um, so I left Berg in 2013. Um, it stopped doing consulting. It started, um, getting VC money to do a product play. Um, didn't work out as many startups. Don't we made a thing called little printer, which some of you might know. Um, 2014, we shut up shop, but everybody from Berg from that studio has gone to all sorts of different things, Apple, Microsoft, making games like Playdeo, ending up at Apple as a result of that. And in my case, I went to Google. So the sort of Google box, if you like, is right up in that corner when stuff started getting very, very, very, very strange, I suppose. So I moved to New York City to work with the folks of a Google Creative Lab who I first worked with on Matt Lamp's project, did a ton of concept and design work, which will never see the light of day.
Some things did make it out there. So a project called Google Sunroof, which was about using Google's map data to map the solar potential of every roof in the world. Still very, very proud of this. This was a tool which was meant to match up homeowners and solar installers and tell them how much they could save, et cetera, et cetera. Worked on the sort of branding and interface and marketing of this with my now partner Kelly, who's in the audience. And this was one of those things when Google was very much doing lots of wild bets and sort of the dawn of Google X and Alphabet and those kinds of things, the sort of Larry and Sergey era. And so there was lots of work in the sort of energy and energy transition space, which never went anywhere. This got out there, but there was loads and loads of stuff, which I wish I could show and wish I could talk about.
2015, moved to Google Research, started working on sort of AI products mainly, looking at sort of UX challenges around kind of AI. And again, it's this kind of problem. You know, the stuff beneath the surface was just getting less and less legible and more and more capable and sort of more and more magical in the wrong way. There's a whole other talk about the harm referring to technology as magical does, but that's for another day. But in terms of the metaphors there, like in my first, I think, kind of two weeks of this work, so moving over to Google Research, there's about like 3,000 computer scientists and about like 30 people doing design or UX research.
And John Giandria, J.G., was the head of that department at that time. He's now at Apple. And his, you know, I sat and had lunch with him, we were talking about it, and he said, like, "Listen, any time you hear somebody say 'artificial intelligence'" — you might not say this these days, but, you know, it's like, "replace it in your head, search and replace it with 'learned systems'." First of all, because that's actually what they are. They're learned rather than coded. And secondly, because it's boring. And so it gets rid of that that kind of weirdly inflated cultural expectation that we have about AI and robotics, et cetera, from all of the cultural artifacts around that, films, books, science fiction, et cetera.
And this gets to the heart of how it actually works. This is the phenomena. This is the material. And Lens Systems kind of got transmitted into a thing called Teachable Machine by colleagues, ex-colleagues of mine at the Creative Lab, Google Creative Lab. It's wonderful. It's still up there. It's a really useful tool, training thing, teaching thing. It's a browser-based, private ML system. It's great. You can teach kids what ML is. It's such a great way to kind of-- like a playable explanation of what a learned system, what ML actually is, and how it works as a sort of stepping stone. And lots of people use it to make projects as well, because you can kind of like spit stuff out the other end as well. It's very, very cool. Me with a pot plant, hold to record, train the model. That's what it all is, just bigger. 2016, started working on physical products like this one.
This is a camera called Clips, which doesn't have a viewfinder. What it does have is a little tiny AI inside it, but it's an edge AI. It's not connected to the cloud. It's on the device. Moving the models to the device just became possible at this time. starts to give you enormous advantages, both in terms of privacy and agency, but also in terms of energy. One of the things about the cloud is that you have to talk to it using radio. Radio is very expensive energetically. Uses a lot of battery. So if you want to have small, wearable things like this, try and make it not talk to the network as much as possible. Doesn't have a viewfinder, but it does have something that looks a little bit like a shutter button. It is a shutter button, but it's also a voting button. It's a training button. It's just, it's the button to teach the teachable machine. And when you take a photo with it, it says, hey, I like what's in front of me right now. And so what gets trained is the model on the device itself.
It's your tiny private model inside that device. And I thought that was such an interesting notion and such a kind of like really sort of like different default to kind of where everything else was going, but I really wanted to sort of look at this and the opportunities of this. And this sort of edge AI is incredibly interesting. 2018, Pete Warden, one of the people who worked on kind of getting all of this stuff down small enough and performant enough to work on device, started to talk about this openly, you know, that's still up there. But what he talks about here is what I was just describing. It's just kind of like incredibly expensive to move data around using energy. If you can do it where the action is, it's going to be cheaper, which means you can make things smaller, and you can start to power them differently.
So again, the difference in the architecture creates different opportunities. Moving away from a default expectation of all of this stuff being in the cloud to making it something more distributed starts to give you different opportunities. Connected to that same group, I wasn't directly involved in this work. A lot of this work was done by other colleagues of mine. But this is a technique called federated learning, which is about connecting together all of these. This is sort of like being able to have your cake and eat it, in a way. So this is about having lots and lots, a fleet of sort of distributed devices that are all doing this on-device edge AI machine learning. But then being able to kind of couple them together in a network, in a privacy-preserving way, an approvably privacy-preserving way to create sort of network effects of learning to improve a model which can then go back down to all of these edge devices when they do connect to the network.
This was sort of, at the time, kind of seen as almost like heresy. This was kind of like, "What the hell are you working on? "What is this?" And now it's kind of a widespread practice across the industry, Meta, Google, Apple, Amazon, all pursuing federated learning, Edge AI models, all of these kinds of things. And we were also working on phones and Android, and we were starting to think about what could you do uniquely with Edge AI on the device. And so Android's sort of like version of Shazam, if you like, was a embedded neural network on the phone, incredibly fast, worked when you didn't have a network, could work in airplane mode, but it was kind of like, and it was also something which started to then speak to the kind of difference in kind of the chips that were starting to go into these devices as well, which was starting to be specifically around accelerating machine learning operations, et cetera.
And so what's kind of interesting about little things like this is it sort of, again, gets away metaphorically from this idea of AI as being something which is about the brain or kind of consciousness or something that replaces humans. So you can have stuff that works using neurons. We have neurons all over our body, right? When somebody hits your knee with a hammer like they used to do in the doctor's office, the neurons in your knee, this kind of like local loop around your spinal cord is making the action happen. So it's kind of, again, sort of like thinking about this kind of like more distributed, embodied, different way of thinking about AI and how it can kind of manifest than the default.
And so, one of the other things that I started working on in around sort of 2018 to 2022, before I left Google, was, again, like moving away from where the default of the industry was, smart speakers, services that you used a hot word to kind of access, which are effectively, they are personalized, but they're centralized. So again, thinking about the architecture of this and thinking about personal AI and sort of extrapolating from these ideas of, in Clips and the ideas of federated learning, to what would be a truly personal AI.
So rather than us all speaking to one thing in the cloud that we have accounts in, these are almost like mass-manufactured AIs. And they're yours. Sort of when you leave the … you know, when you sort of like drive it out of the showroom, it's yours. It's not Google's, it's not Apple's, it's not Amazon's. And so, you know, the idea of like having this distinct line between your personal AI, the user, and then the tools and systems that this team would then work on.
So, you know, these would both be perceiving and working upon shared surfaces. So your personal AI could speak to Google for you or with you, but it wouldn't be Google. It wouldn't be Apple. And again, this is a sort of architectural choice that we can kind of make. It's entirely possible. We just choose not to do it because of the metaphors and the business models and the way that things are working right now. But, you know, a truly personal AI should let you break into a space station, right? It should be on your side, no matter what you want to do. And so we did a bunch of work about what would that mean? So, you know, what would it be like when you sort of drive it out of the showroom? What would it be like to have that kind of unboxing of a personal AI and training it? We worked with a London-based studio called Special Projects on that, and thinking about this idea of creating safe spaces and props and a practice arena, almost like the training levels of a video game, so that you can start to be confident about this little R2 unit that you just acquired.
We also started thinking about two-player control surfaces. So one player is human, the other player, if you like, is the AI. So this is a sort of keyboard of sorts, a sort of live visualization, where you and your personal AI are working in a search space. But it's constantly redrawing these kinds of things based on your action, what it's found, and its notion of what it knows about or what it's learned about you. And we built this with colleagues in Google Research, and also a small studio in London called Nord Projects. Moving on from that, last projects at Google were kind of more exploration in this space. Again, trying to get away from the dominant metaphor of dialogue, of conversation, into kind of multiplayer graphical user interfaces, rather than command-line interfaces, or text or speech conversation.
And this idea that you could have more like magic canvases than magic-trapped humans in speakers that you had to speak to. And it's kind of interesting to think about this in the current frame of things like Midjourney and Dall•e and ChatGPT. Again, they are still kind of like … A lot of them are kind of like turn-taking. I think what Midjourney and some of the things which are layers on top of that are kind of interesting about it is they've started to turn it more into a kind of like multiplayer game. And I think that becomes really, really interesting.
And there's a small startup in Barcelona called Firma that has sort of been pursuing this with sort of a sort of supercharged version of kind of like Miro or Mural or something like that, infinite canvases, but then also have like agents embedded into them, which are generating things working with you. And so that sort of leads me to the kind of the switch in a way.
2022 I left Google, started work for Lunar Energy. As I mentioned, we're an energy tech company. We make software and hardware to try and move people over to clean electricity in their homes, primarily. And sort of, you know, instead of overlaying Moore's Law on my career, I sort of overlay this slightly more, well, vastly more depressing visualization, which is Ed Hawkins' famous visualization of climate stripes and kind of how, you know, global heating has kind of progressed over that period is kind of, just again, like, I know why I want to do this. So we make solar battery systems, but we also make software. So we make stuff which is kind of connecting the cloud and the grid.
And sort of one of the things that kind of was part of my journey to sort of coming to this was this book by Saul Griffith about kind of what we will need to do in terms of moving the needle in terms of the climate emergency. And so much of it is about kind of moving over to electricity instead of digging stuff up and burning it. So what's the role of a designer in this?
I've been thinking a lot about something that Matt Webb used to talk about back at Berg, which is there are four types of product and who doesn't like a two by two, right? So, there are despair products that you kind of have to buy, like toilet paper. I guess you don't have to, but … There are fear products that you buy because you're afraid of something might happen. You know, I don't know, burger alarms, or perhaps even like insurance. There are greed products that you buy because you think they're gonna get you more of something. So that might be kind of like fashion, to get you more attention, or it might be an MBA, or a degree at AHO to get you more success in life, or an IxDA membership.
And then finally, and the thing that we always tried to, we tried to use this taxonomy to challenge ourselves and also in our consulting work. Finally, they're kind of like hope products. And hope products are the things where you kind of like pin your aspirations to kind of like how you want to change or how you want to change the world around them. And they're always kind of vastly successful in the marketplace. Nike is a hope product in a way, right? You buy Nike trainers because you hope you might be like the athletes that wear Nike trainers, but also, you're embedded in a culture that you hope you might impress people in, or whatever it might be. But how do you do this around energy? There are fear products, "I'll be okay when the apocalypse comes." or despair products, you know, a lot of those are a little bit about kind of like, "Reduce your consumption, reduce my bills," those kinds of things.
Associated with that might be kind of, you know, "I could make more money out of this, I could save money, I could maybe even sell things back to the grid," and all of these are good motivations. But we haven't really kind of come to that kind of notion of a hope brand or a hope product in the world of energy, I'm really interested in what that work is and how we might do that. And how can design and designers move energy from despair to hope? And climate change as well.
And there's a very long quote which I won't read because we're running out of time. But it's from Kim Stanley Robinson, the science fiction author, and talking about kind of the roles of utopias and dystopias, but also the fact that in design work or in proposal work, you shouldn't just, like, create utopias. You should create things which are, like, anti-anti-utopia, because people create things which are anti-utopian all the time. "We shouldn't do this. We should keep doing this. "We shouldn't move that." People create dystopias all the time. People create utopias all the time, but what is an anti-anti-utopia? What is the thing which is so good at answering these arguments through what it does in the world and how is it explained that it really does move the conversation on?
In 1964, a physicist called Nikolai Kardashev created a scale of civilizations based on their ability to harness energy. So type one Kardashev civilization is defined that as a civilization we can harness all of the energy that reaches its home planet from its home star and Then you go on to like, you know type two and type three. We won't go there because we're not you know, we're not at We're not at type one yet
But a type one future could be amazing and the best piece of work I've seen around this is not from a tech company or an energy company. It's from a yogurt company: "This place is yours now. It's a handful, but nothing worth doing is easy. The land is more than just dirt. If you look after it, it will feed you forever. You're a smart one. I know you'll be okay, come rain or shine. And remember, a business is only as good as its people. So treat them well. Our job is to plant seeds so our grandkids get to enjoy the fruit."
And so I've been reading a lot of, I mean, that's amazing. And like, you know, I wish I had like the ability to create studio Ghibli animations about my projects. But I've been reading a lot about these kind of like more hopeful futures. You know, solar punk is like a big movement, where you can read so many brilliant, inspiring, anti-Utopian narratives around this stuff.
And then there's also kind of like this one, which I've kind of been really loving from Tobias Revell, who's a speculative designer and educator who works at Arup in London, which he's sort of called "breezepunk". And it's kind of based out of this kind of technological innovation which sort of harnesses the slightest breeze to sort of create and store electricity very very tiny amounts of energy. But then if you sort of like team that up with something like ai at the edge It starts to get us to something really really interesting So back to the present.
Um Currently, we're at a sort of 0.7 we think maybe uh of a KIardashev score. But we're like four orders of magnitude more energy possible to harness from the solar potential of Earth. This is a tweet from Simon Evans From earlier near January sort of indicating that we're kind of at this tipping point from moving from Fossil fuel from digging things up and burning them to kind of power power our world. And then finally also came out earlier this year this book called No Miracles Needed Which is all about the fact that we have everything we need technologically.
It's almost in that kind of like, you know, what just got cheap and boring kind of piece that we were talking about we just need the kind of will to build it up and connect it and sort of create a sort of different deliberate machine around that so coming back, you know switching over to the grid.
You know, we think of this as like big physical interest is I'm from Wales. This is in North Wales, it's a massive battery made out of mountains and water and whenever British people make tea which is a lot, they switch on kettles. This is always really hard to show to Americans because they don't have kept electric kettles in America. It's really really hard to get one if you move to America And so, when we think of electricity, we think of big infrastructure, physical infrastructure.
There was a great report in The Economist recently, which sort of pointed out that the grid is not just this. It's oddly immaterial and highly dynamic, because the truth at the heart of the grid is that supply has to match demand in real time. I won't.
I'll skip beyond this next, this next, this is an explainer which was part of that report in the Economist. I'll ask you to look at it yourself. But it's all about this kind of, you know, it really nicely unpacks the material qualities of the grid and why it's so important that it operates at a rhythm and what the sort of grains of it are. And what really is happening is that software is eating this, it's changing this.
So the cloud is starting to affect the grid. And this is a diagram from an EU report a few years old, which is about kind of the changes that are happening across these layers, these pace layers, which sort of particularly are sort of fundamentally changing down at this bottom layer of the end consumer, the homeowner, you and me.
We're going from being a passive, just paying sort of, you know, despair. Energy is a despair product to something where we can be actively participating in the production and the distribution and the transmission of energy, and a big part of that is about storage and software batteries and the things that connect batteries together.
This is a visualization of Work that we've been doing It's kind of hard to see unfortunately in this light. But this is the Bay Area in California on one day in the autumn the orange bars are batteries being filled up by the sun. So this is solar energy going into individual households.
Now what we'd then do is we start to connect these together with software to turn them into something called a virtual power plant. So this is almost like combining all of these little packets together all of these kinds of things into something which is more like a a sort of an internet of energy and then you can start to coordinate its release at night time to the grid. So you can start to have solar and wind these intermittent sources of energy which are generated at that kind of decentralized level come together in a centralized way and show up on the grid.
This is another one of Japan. It's very hard to see unfortunately, but this is sort of really nice because it's kind of this almost this is a day of all of these batteries of Japan storing and Releasing energy almost like you know inhaling and exhaling, you know, it's almost like a natural rhythm. It sort of feels like a Hyper object breathing which is I think is you know, and and sort of finally this is What could you really do with that which is kind of this is if you have a layer of software which knows how to connect to other software, i.e. reports about typhoons and hurricanes, you can start to change the behavior of this decentralized mesh of storage, of energy.
So what you see in this is, as the typhoon is starting to approach the tip of Japan, the batteries are starting to go, "Alright, I'm going to start saving up some energy to be able to deal with this in case the grid goes out. So you start to get something which is not only clean but more resilient more intelligent more able to serve be responding to both local and and national and global conditions.
Final sort of recommendation about this is this book downtime on the micro grid by Malcolm McCullough And what he talks about in this, I just sort of highlight this passage, is this idea of a new grid awareness and that this could be delightful. It could be making it evident, making it beautiful, could really, really work.
Not just, you know, rather than it being something that we try and make a service that you just pay for and it goes away. What are the sort of opportunities here, again, through that sort of lens of how we look to work in Berg, to start to think about the energy transition and start to think about kind of how you might think about embedding these scales within each other, making those interfaces evident to the end user so they understand them, to kind of like build these kind of layers of sort of what Benjamin Bratten calls planetary wisdom or sapience, because we've kind of, you know, we've done it unintentionally, now we have to do it deliberately.
So I think this still works, please do not make the metaverse. Make the anti-metaverse, the alternative to the metaverse, this thing which runs at pretty high resolution, better and survivable with what you design and build.
And thank you very much.
About the speaker
Matt Jones is Principal Designer at Lunar Energy, a renewable energy tech company with offices in Mountain View, USA and London UK. He works across all brand, product and experience design there.
He is a designer with a broad range of experience delivering both digital and physical products/services since 1995.
From 2009 to 2013 he was a principal and partner at BERG, a product invention company in London that spearheaded the development of cloud-connected products with exemplars like Little Printer, Cloudwash and Mag+.
Through the experimental projects he led at BERG he helped invent new ways of interacting with the digital layer of the world around us.
His work at Google continued this theme, with projects that showed what can be achieved by bringing together experts in art, technology and design.
Matt was one of the founders and lead designer of Dopplr.com, a service for frequent travellers which was sold to Nokia in 2009. At Nokia, he drove design research, working on areas as diverse as RFID/NFC applications of tangible/physical interfaces and the human universal experience of play. Then, transferring that work to the business-facing side of design — as Director of User-Experience Design for the Multimedia Division of Nokia Design, part of the team responsible for driving the Nokia Nseries user-experience.
He has also been a creative director and team-builder for award-winning services like BBC News Online, and Sapient's London studio. Matt teaches and lectures at the RCA, AHO Oslo, CIID Copenhagen and Goldsmiths, London. His writings on interaction design, comic books and planetary-scale, self-replicating robot dogs can be found at Petaflotptimism.com
About Lunar Energy
Lunar Energy was founded to transition homes to 100% clean energy—making our electricity greener, our air cleaner and our energy more safe, secure and reliable for all.