The fourth episode of A4N — the Artificial Neural Network News Network podcast — is out (listen on my website, on Apple Podcasts, Spotify, Google Podcasts, or YouTube). In this episode, our guest host Rasmus Rothe joins us to discuss Merantix, his rapidly growing AI Venture Studio, and how they are applying machine learning to revolutionize cancer detection, self-driving cars, and more.
Dr. Rasmus Rothe is a German native, and co-founder of Merantix, the world’s first AI-focused venture studio. Merantix has already launched three successful AI-driven companies with three more operating in stealth, and raised an additional EUR 25 MM in 2020 to continue to apply world-class AI research to solving practical issues. Rasmus published 15+ papers on deep learning while attending Oxford, Princeton and ETH Zurich, where he received his Ph.D. in computer vision and deep learning. Before founding Merantix, Rasmus worked for BCG, Google, and built a deep learning service with 150m+ users. He is also a founding board member of the German AI Association.
Reference links:
2:28 Merantix
5:27 Deep Learning Illustrated
14:35 Merantix raised 25m euros
20:00 Vara Healthcare
20:10 Vara raised 6.5m euros ($7m) in Series A venture capital
31:41 Siasearch
38:50 Dr. Alex Flint's start-up, Zippy, was acquired by General Motors' self-driving car unit in 2018
41:30 German A.I. Association
46:20 Dr. Rasmus Rothe's LinkedIn
46:28 Rasmus on Twitter
47:15 Jon Krohn on LinkedIn
47:40 Jon Krohn's email newsletter signup on his homepage
48:00 Jon on Twitter
Transcript
Jon Krohn 00:06
Welcome to A4N: Artificial Neural Network News Network, the show about the latest developments in artificial intelligence, machine learning, and data science, where we both introduce technical aspects of these advances, as well as discuss their social implications. In today's exciting episode, we'll be discussing self-driving cars, how machine learning is revolutionizing radiology through early identification of tumors, and AI venture studios with Dr. Rasmus Rothe, the founder of the world's first and most prominent AI venture studio. My name is Jon Krohn, and I'm your host for today's program. Let's get started. So, Rasmus, where are you joining us from today and how are things going for you under the lockdown?
Rasmus Rothe 00:50
First of all, thanks, Jon, for the invite. I'm right now in Berlin and I'm actually at the office right now. We've been fully remote for two months at Merantix and then a few weeks ago, we started to open up the office again, and right now we are on a kind of rotation. So, around a third or so is at the office right now.
Jon Krohn 01:08
Wow.
Rasmus Rothe 01:08
So, slowly getting back to normal, hopefully.
Jon Krohn 01:11
What was the extent of the lockdown out there in Berlin? Was it completely shut down at some point?
Rasmus Rothe 01:17
Yeah, it was completely shut down, streets were empty, all the shops and restaurants were closed. But now, as of a few weeks ago, things opened up again. Numbers are not going up too crazy. So, I guess this will be the new normal for a couple of months until hopefully everything is fully back to normal. Let's see. I'm, I'm also curious what's going to happen.
Jon Krohn 01:41
Nice. Well, I'm glad it sounds like infection rates there in Berlin are staying down despite the reopening. So, hopefully, that's a good sign for the rest of us in places like New York where I am, where we're just finally, today, ending 100 days under lockdown. So, in our previous episode of A4N, we mentioned you with respect to a number of items, including your racy Ph. D. research. So, not something we can say about much machine learning research, but in this case, I think it does apply because in its first month alone, people from around the world visited howhot.io to pass 50 million photos into a model you designed in order to assess the subject in the photos age, gender, and attractiveness. We're also going to discuss Merantix, in fact, this will be the focus of the episode today and lots of news around Merantix, and its related startups. Merantix is an AI venture studio, which recently received substantial funding. So, we'll talk about that news, as well as what AI ventures studios are. So, let's start off with how you and I met and then you can lead us from there through to your Ph. D.
Rasmus Rothe 02:52
Sounds good. Yeah. We actually met back at Oxford when I guess you were doing your Ph. D. and I was an undergrad and we worked together at Oxford Entrepreneurs trying to get the Entrepreneurship Society of the Ground at Oxford. It was super exciting. I guess not much machine learning, at least in our work at Oxford Entrepreneurs, but still, the topic of entrepreneurship we were focusing on. So, that was great.
Jon Krohn 03:18
Yeah, exactly. You've definitely been a poster child from that kind of starting point where being in an entrepreneurship society as an undergrad taking what we could learn from the various workshops that we ran, and the inspirational kinds of speakers that we had, and going off and creating your own startup, I guess, during your Ph. D. Is that right? Or just afterwards?
Rasmus Rothe 03:44
Exactly. Yeah, it was right after my Ph. D. I hadn't defended my Ph. D. at that point yet, but I was already done with all the publications I had to do. So, I had a couple of months kind of preparing the next step. And during that, I actually met my co-founder and then we started working on Merantix for a couple of months.
Jon Krohn 04:00
That's nice. So, tell us about your Ph. D. research. I mentioned howhot.io. That's probably not the part that you want to focus on. Or maybe it is! But I remember that was big news when it happened.
Rasmus Rothe 04:11
Yeah, for sure. Basically, after Oxford, I have spent some time in the US and then move to ETH Zurich to do my Ph. D. with Professor Luc Van Gool. And in my Ph. D., I actually focused on computer vision, various computer applications, and also using neural networks. So, I was-I was doing my Ph. D. between 2013 and 2016, so I guess it was right when the Alex Krizhevsky paper came out from ImageNet. So, people started to try to apply neural networks to all sorts of applications of computer vision. So, I wrote a few papers on face recognition, inferring age and other facial traits from faces, but I also worked on completely different things. I worked a bit on super-resolution, which is around upscaling images to a higher resolution using machine learning there to improve quality. A bunch of random topics. I was actually, I think the first or one of the first people in Luke's lab is using neural networks. So, before that, I was focused more on classical computer vision. But in 2013, people started to get excited about neural networks, so I thought, "Okay, this is interesting. Let's try it out." and I saw a group just works pretty well. So, that's when my Ph. D. focused on this.
Jon Krohn 05:26
And for listeners that aren't aware, it's actually the main focus in my first book, Deep Learning Illustrated Chapter One. It's all about how in 2012, these researchers at the University of Toronto in a lab led by one of the most famous deep learning researchers, we're going to talk about him again later in this episode, Geoff Hinton, someone in Geoff Hinton's lab named Alex Krizhevsky, which Rasmus already mentioned, released this AlexNet Architecture that absolutely crushed all existing benchmarks for machine vision in 2012. And then what a time for you to be doing your Ph. D. starting 2013, to take notice, like anybody working in computer vision would have to have and say, "Wow! This is something new,” using these neural networks, these deep learning architectures for machine vision. How exciting to be able to apply those to many different applications like you have. I'm super jealous that you have that timing perfect. And was it the, “Some Like It Hot” paper, was that the biggest paper that came out in your Ph. D.? That's what we talked about last week. It was this, “Some Like It Hot: Visual Guidance for Preference Prediction.” But you had a bunch of papers in your Ph. D.
Rasmus Rothe 06:34
Yeah, I think it's not the paper with the most citations, but it's certainly the one which got the most media attention. The way it happened is I was setting up a collaboration with a dating app I met at an entrepreneurship event. We then decided to train some neural networks on that data. And basically at the end of the project and the paper I wrote, we then said, "Okay, let's build a little demo website so people can also try out the algorithm and upload their own photo, and get a prediction around their age, their gender, and their facial attractiveness according to the data set." Then that website just went surprisingly viral. So, we just thought it would attract a few geeky researchers who find the algorithm interesting. But basically, after a couple of hours, there were already quite a few users on this website. It was a big surprise.
Jon Krohn 07:24
Yeah. If I remember correctly, you mentioned when you were visiting New York a few years ago that the cost of maintaining the website, all of a sudden became substantial, right? Did you get sponsorship for it?
Rasmus Rothe 07:37
Yeah. We first hosted it at the university and then when I shared it with some friends, the server went down. So, I was like, "Okay, let's set it up in a bit more scalable way." So, we hosted it on AWS and we basically had to call them every hour to get more instances and kind of increase our quota. That went crazy. We didn't know how to pay the bills, we just kept it running. Then a few days before the bill came in, AWS supported us. And in return, we also wrote up a case study about this crazy scaling experience. So, I think it was good for all sides at the end.
Jon Krohn 08:19
Yeah. That is, to reiterate the stat that I gave at the beginning of this podcast, there were 50 million photos, 50 million queries made just in the first month. That was exciting. I remember that was something that came across me organically. It was such a big splash in our field. I think maybe even outside of machine learning, lots of people were just throwing their photos in to see or other people's photos in and seeing how attractive they rated them as. And then I noticed through that, I was like, "Oh, I know–I recognize Rasmus' name here." So, very interesting. Then at the end of your Ph. D., you say that you had some time at the end there. What a fortunate situation to be in that you have some extra months after writing all of your Ph. D. papers to figure things out. So, it was in that time that you met your co-founder? Was it Ph. D. research that led directly to your startup or was it orthogonal?
Rasmus Rothe 09:17
I think it was inspired by Ph. D. research. I mean, my Ph. D. was really focused on applying machine learning or deep learning to some areas like face recognition where it wasn't really applied much before. So, I certainly realized during my Ph. D. that in every single industry and every single application where data is available, machine learning will play a very vital role. But at the same time, I saw the challenge of, even if you can just build an algorithm–and that is obviously getting easier and easier–that is still very far from actually having a real company in a real application. Just because you can build an algorithm doesn't mean you actually have the access to the data sets you need, or you have embedded the algorithm into a product that saw some users. So, there's much more to it in building an AI first company than just building an algorithm. I saw quite a lot of companies, which were very much focusing on the AI part only, but my big belief is that that’s only like 5%, basically. So, with my co-founder then, I was exploring to figure out what is the way to basically enable all these AI-first companies and make them more successful so they can really become a successful large company. That's where the idea of Merantix was born , to basically build a venture studio that serves this purpose.
Jon Krohn 10:39
Nice. I didn't know that. So, the point of Merantix from the beginning was to have this venture studio concept.
Rasmus Rothe 10:45
Yeah, it was, I mean, we didn't know the venture studio model very well. Actually, funnily, on some slides from 2016, it says, "Venture studios as one way how we imagine to build Merantix," but it wasn't like us saying, "Okay, we build a venture studio." it's more like, what we built, and we only really realized that last year, is actually a venture studio. So, in retrospect, that's what we've been doing for the last four years, even though we only started calling ourselves this starting last year.
Jon Krohn 11:17
Nice. We'll get into details of exactly what an adventure studio is momentarily, but just before we get there, so when you started Merantix, what was the business model? What were the AI applications that you had to industry?
Rasmus Rothe 11:29
In the first six months when we started Merantix, we just did a lot of interviews. We talked to around probably like 100 companies and asked them about their challenges, where they have data, and where they see potential. In return, we actually explained to them why AI is important. I guess now it's much more all over popular media, but in 2016, many people from the management level still wouldn't know that much about AI. So, it was kind of, “We tell you about AI and you tell us about your data opportunities”.That's when we actually realized that there are so many different opportunities out there, but at the same time, similar challenges, like getting access to data, fitting into existing workflows that we said, "Okay, we build Merantix as this kind of platform that builds AI applications, ultimately as its own companies." So, we then just started to hire the first couple of people and started to work on the first applications, which then later became the first companies of Merantix, like Vara in healthcare and Siasearch in the space of automotive, for example.
Jon Krohn 12:28
Excellent. We're going to talk about those examples in detail momentarily, because they are both making a splash, and both are highly relevant too to emerging AI fields. So, Vara Healthcare is transforming radiology in medicine, and Siasearch is transforming how data are used in self-driving cars. We're going to talk about both of those concepts in detail. But tell us what else has been incubated now in this Merantix venture studio and tell us also what exactly an AI venture studio is. How is that different from an incubator, for example?
Rasmus Rothe 13:14
That's a good question. Venture studio is a sort of incubator, but much more focused on one theme, in our case, AI. It also tends to focus on building less companies in the same time. So, you have some of these company builder models who have been building ten, twenty, thirty companies at the same time. A venture studio is a much more bespoke model where it's really only about building maybe one, two, three companies per year, but with much more dedication and focus, and also much more capital, ultimately, behind those companies. That is one part. Another part, which is very special about a venture studio is that, in company builders, you tend to have a lot of centralized services, which you provide to all your startups you incubate, and in a venture studio, you tend to not have very much centralized services. You actually tried to build really independent companies, but rather through a common brand and maybe colocation, and maybe some tech infrastructure and shared knowledge you create ultimately a benefit for your companies. So, all the engineers, all the product teams, and everything is part of the companies we incubate. It's not centralized.
Jon Krohn 14:27
Nice. That is a super cool idea. It seems to be taking off. You guys raised 25 million euros or about 27 million US dollars to expand your AI venture studio in January of this year, of 2020. You guys were the first. As far as I'm aware, you're the most prominent of this kind of concept. So, when you're coming up with these company ideas, One to three companies a year, how does that happen? Do you take applications for people's ideas? Or is it more top down, were you and other existing companies or executives come up with some concepts and then find a team to build that out? How did you come up with the subsidiaries of the venture studio?
Rasmus Rothe 15:14
Yeah, I think this is the million-dollar question. The core of what we do is that we bring entrepreneurs-in-residence on board. We basically hire people who want to start companies with us and in the first six to twelve months, when they are basically still part of Merantix, we ideate and also validate a lot of potential business ideas with them until we then spin out an actual company with them and then become the CEO or CTO also, which we then also fund and kind of scale to Series A and beyond. The ideas actually come from very different parts. Sometimes entrepreneurs-in-residence bring ideas or areas of their expertise with them, and then when they come on board, so somebody might have a very deep background in a certain industry because the person has worked there before or was in academia in some space, and is very, I don't know, knowledgeable about certain technology, but that's only one part of the equation. I think the other part is that we also actually maintain relationships with a lot of VCs out there with Merantix Labs or one of our companies, which is our solutions provider. We also run a lot of consulting projects and service projects with large and small companies. We also learned a lot there where people, where opportunities may lie. I think it's, in the end, a combination of Adrian, my co-founder, and me, seeing a lot of stuff, all the activities within Merantix, but also what the entrepreneurs-in-residence ultimately care about. And that's how the ideas get formed. So, yeah, it's pretty interesting. And we definitely put more and more structure on this process and how we are evaluating an idea–here are also a lot of things we have discarded for one reason or the other. I think this knowledge base helps, obviously, when you look at new ideas.
Jon Krohn 17:05
Well, I love this concept and I have no doubt that the early success you had with it, Rasmus, is going to continue. It's really inspiring. I love the way that you've set it out and I think great things are to come. We're going to talk about Vara Healthcare and Siasearch in detail next. But are there any other firms that you want to talk about right now that are under the umbrella of the venture studio?
Rasmus Rothe 17:25
Yeah, I can't say much. We have a few companies we are right now working on, which are pretty much in stealth and like also very early, so teams are still very small. One space we're particularly excited about, and I guess it's also related to your background actually, Jon, is the intersection of biology and machine learning. That's actually one area where we're now starting a company. Biology is certainly creating more and more data, for example, with sequencing costs going down, but also in other areas of biology. So, I think there's a lot of very interesting applications at the intersection of machine learning and biology using that data and making sense of it. That's one of the areas we're right now starting a company. Another area is a more horizontal topic. It's focused around analyzing a lot of data that is produced in companies. So, it's in the space of business intelligence and kind of automating parts of the work of a BI analyst. Still also, very early, we're already working with a few customers, but the company is just at the beginning.
Jon Krohn 18:32
Nice.
Rasmus Rothe 18:35
And then there are a few other areas we're looking into. We find the data privacy and models around that also was related to machine learning quite interesting. We're also looking at some things in the space of manufacturing. Pretty broad. We're completely industry agnostic and technology agnostic. Given my background, historically, we've been focusing a lot on computer vision, but now we are also gearing more towards text language data, NLU, NLP, because obviously, there has been a lot of progress in the last one or two years from the technological side which opens up a lot of very interesting applications, but then also other types of data like biological data, not necessarily image or text, but kind of maybe some other structure of data. I think those areas are also super exciting.
Jon Krohn 19:18
Brilliant. Yeah, that sounds like a great mix. The horizontal play makes a lot of sense to me. Business intelligence tools, there are lots of tools that do things like this, of course, but if you can find a specific niche, find clients that are willing to pay for that specific new thing that you're developing, then this is definitely something that can be big. Of course, yeah, I'm excited about any kind of a vertical use of natural language processing, which is my focus today, as well as the biology stuff. It used to be my focus when I was doing my Ph. D. So, that's wonderful, Rasmus. All right. Let's move on. I've mentioned it a couple of times, the specific firms, Vara Healthcare and Siasearch. Let's talk about Vara Healthcare first. This is a machine learning news show and I've deliberately brought you on at this time to fill in on some breaking advancements in this field, in radiology. Not only has the Merantix AI venture studio been making TechCrunch headlines as recently as a couple of months ago, but Vara Healthcare, this Merantix subsidiary of yours, a breast cancer screening platform, was making headlines just last week, in early June 2020, when it raised 6.5 million euros, so about 7 million US dollars, in Series A venture capital. So, let me frame the problem here. According to the World Health Organization, breast cancer is the most common type of cancer among women with 2.1 million women being diagnosed each year. Early detection is critical here, where stage one patients see a really high survival rate. In cases where the breast cancer is caught early, survival rates are very high. But in stage four, which is the most advanced cancer stage, if the cancer is caught in stage four, survival chances are quite low, an unfortunate 15%. So, early detection is key and the process of analyzing mammograms is time consuming and repetitive. So, there's an opportunity here for automation where manual assessment isn't this kind of bottleneck on getting that early detection, right? So, tell us a bit, from your perspective, about the state of radiology today and how solutions, like Vara's, are revolutionizing outcomes for patients.
Rasmus Rothe 21:37
Sure! Thanks. That was a great, great summary, Jon of the state of this space. I think it's a very exciting space for machine learning, especially computer vision, because you have a ton of data and you have a task that is highly critical, as you mentioned, because if in the case of breast cancer, for example, if the cancer is detected early, its survival rate is really high. But if you miss it, it can be very deadly. It's also an area where doctors, because there are under lot of stress, but also because it's actually cognitively a very difficult task to analyze these mammograms, that this makes actually a very good application for machine learning because in the end, our algorithm–and with Vara we’ve basically built this breast cancer platform, which has been learning from millions of mammograms, to basically take which ones are cancerous and which ones are not, and thereby basically help the doctors to spend much less time reading the exams and also help them to actually miss much less. Because right now, around one in five cancers, which could have been seen, are missed. So, it's very bad and obviously algorithms can do much better. We've been busy building this whole workflow solution which doctors can use to prioritize their readings which help them to look at the images and also can highlight what regions could be relevant, but then also help doctors with the reporting. It's not just an algorithm, it's actually an end-to-end workflow, where we work right now with radiology centers in Germany, but also in quite a few other European countries. Last year, we certified this medical product. So, we now are very much in distribution mode, trying to get more and more centers on board, but also further improving the platform and improving the algorithms and all the support for the doctors around the solution.
Jon Krohn 23:36
Brilliant. I love it. So, one of the key things here is false negatives, right? Physicians, if you're constantly seeing images, maybe it's been a while since you've had a coffee, or had your lunch. We know in all kinds of industries, human fatigue leads to missing things. So, with an algorithm like this, the algorithm can highlight, "Hey. There might be something here, take a close look," and this draws the radiologist's attention to a particular part of an image and then the radiologist can use their years of expertise and training to follow up, maybe do some extra scans, or do a tissue autopsy. So, this makes perfect sense to me and it's brilliant how you've tied that algorithm idea into the broader workflow, which of course, is critical to getting people on board and wanting to use your technology. As you mentioned, having the right algorithm is 5% of it. I like that percentage. I think that's about right.
Rasmus Rothe 24:40
Yeah, I think it’s also a good example for a company we would like to build with Merantix. There's the algorithmic part which is incredibly hard because it's really about making them robust and dealing with all kinds of edge cases, but it's also about building partnerships with dozens of hospitals to share the data in a GDPR-compliant way. It's about building workflows that doctors actually like using and actually helps them in their daily jobs. It's around certifying AI products and working with regulation on AI. So, I think all these different parts we like to bring together, which ultimately only will create a positive impact because it's super easy these days to train an algorithm on some publicly available data sets, you can probably do it within a few hours, but getting this medical product out there, which has obviously a much larger impact is totally different piece, but ultimately important so that we can all benefit from this.
Jon Krohn 25:37
Right. We can all benefit from it in terms of a healthcare perspective, but also commercially. It makes a lot of sense to me because you build–by dealing with all those aspects of it, certification and adoption, you're building a large defensive moat around the company as well. So, that works out really well. In recent years–this might be controversial. I have no idea what you're going to say to this, but in recent years, with the advancements of convolutional neural networks, like the AlexNet Architecture that we talked about at the beginning of the program, that made a big splash right before your Ph. D., two of the most prominent experts in machine learning made predictions that might send chills down physicians' spines. Geoff Hinton, we already mentioned him, he's the guy who ran the lab that created that AlexNet Architecture that completely transformed the world of machine vision. I would say it definitely transformed the world of machine learning. It brought deep learning to so many people's awareness and it really transformed the world in many ways. So, this AlexNet technology was huge, and Geoff Hinton had been working on deep learning-related research for decades, him and Yoshua Bengio, and Yann LeCun, they were jointly awarded last year the Turing Award, which is like the Nobel Prize–it's the equivalent of the Nobel Prize for computer science–for their work on deep learning. So, many people call Geoff Hinton, the godfather of deep learning, and in 2016, he said, this is a quote, "It's quite obvious that we should stop training radiologists. And this is because the deep learning algorithms have become so powerful, so capable, of detecting tumor types." And because it takes so long to train radiologists, he says, "Look, it's going to take 10-15 years to train a radiologist when they start medical school through to having a specialization. We should just stop doing that now." Then a year later, Andrew Ng, who is maybe the most famous machine learning practitioner of all–he founded Coursera and has countless papers and patents behind him–in November 2017, he said something similar. He said, "Radiologists should be worried about their jobs." So, do you have any thoughts on this yourself?
Rasmus Rothe 27:50
Yeah. I'm a big believer of the hybrid model. So, at least for the next I guess, decade, it's clear that for some types of radiological tasks, a machine is just better in terms of the precision and recall. But still, ultimately, there might be some edge cases where the machine is not 100% sure what it really is. It's then great to have this hybrid model where you would then forward the image actually to the doctor and have them have the final judgment. So, I think not using AI at all would be not the best for the patient, but probably directly switching to just AI is also a bit too extreme. So, I think, at least for the next couple of years, we will have a very much hybrid approach. I mean, in the end, there's also a lot of other work that radiologists do, right?
Rasmus Rothe 28:38
All the interaction with the patients and then figuring out what are the next steps, running screening centers or the radiology practices. I think they still have other tasks which will not be automated by AI in the near future. So, radiologists will be still around. At the same time, if I were to study or do a medical degree today and think about what I want to do for the next 40 years, I would maybe not focus on radiology. If you look at it on a 30 to 40-year time horizon, radiology might work very differently than it works today. So, yeah. I'm very optimistic that that's going to happen. But it takes some time. I mean, it's a very regulated industry for a good reason. It can be very critical. So, it will take a few years until there's really adoption at scale.
Jon Krohn 29:33
Yeah, this makes perfect sense to me. There's so much more to being a physician than being able to identify some aberration in an image and AI is not going to replace that anytime soon. And of course, AI is really, at least with the kinds of approaches we have today, like deep learning, it really is only effective in situations where the issue is similar to common issues in the training dataset. So, edge cases, things that are out of the ordinary, the machine learning algorithm probably won't notice at all. So, there's this huge gap so that the machine vision algorithm can outperform the radiologist in situations where it's a well-defined tumor type in a common type of situation where there were lots of training examples, but it might be completely blind to unusual situations, which_the unusual situations add up. So, from just being able to recognize aberrant tissues, like you say, there are algorithms that are outperforming in some situations, but still we need radiologists for that piece. And then for the broader piece, of course, we're going to need a radiologist for some time. I mean, even just for liability, right? The legal questions alone, even if we had, somehow, some algorithm that could handle most aspects of a radiologist job, which we are nowhere near, we would still have this big legal issue around liability. So, even for that reason alone, I don't think radiologists will be out of work for, as you say, at least a decade or so.
Rasmus Rothe 31:33
Yeah. Exactly.
Jon Krohn 31:35
Segueing now to another machine vision topic, this one is autonomous vehicles. Self-driving cars. Another Merantix portfolio company is a prominent and quickly growing player in this space. This company is called Siasearch. And let me frame the issue here. So, for decades, we've had motor vehicles with automatic transmissions. You didn't have to change gears manually. But other than that, car driving had been completely manual in terms of pressing the gas and the brake and turning the steering column. In the past few years, however, we first started to see cars that could parallel park themselves and that seemed like a nice little novelty. And then Tesla, the world's best-selling electric carmaker, which produced its millionth vehicle just in March 2020, released its Autopilot feature, which five years ago started enabling folks to drive hands free on the highway. Teslas however, are limited to self-driving on the highway, and they may not be equipped with enough hardware such as LIDAR, the laser-based type of sonar, to handle navigating in urban conditions. So, Tesla's Autopilot: great on highways probably can't do cities. In 2016, however, a firm called nuTonomy launched the world's first self-driving taxi service in Singapore and now several major corporations ranging from tech giants like Google with its Waymo subsidiaries little steering–wheelless vehicle, which has driven millions and millions of miles, to ride hailing firms like Uber to a slew of traditional automotive companies who would like to ensure they aren't left behind, we have tons of companies that are now launching completely self-driving vehicles. So, in order to train all of these autonomous vehicles to be safe and reliable, we need tons of data. The vehicles all collect reams and reams and reams of data about a terabyte of it every day. So, this is enough to fill a typical modern laptop every single day collected by every single modern pre-autonomous vehicle that's out there. So, the millionth Tesla vehicle that was released in March, each of those million vehicles is out there recording a terabyte of data every day across a slew of cameras, lasers, and sonar sensors, and this then presents a problem. So, these data are completely unlabeled. It's just a giant stream of information. It's also often completely unstructured. So, there could be tons of valuable data being collected, but the typical autonomous driving firm makes use of only 5%, 1/20 of all of the data that's being collected. All right? So, that frames our problem here. And Siasearch, this Merantix AI venture studio portfolio company, it sounds like you guys are working on the solution, right?
Rasmus Rothe 34:41
Thanks for the great problem description, Jon. Yeah, that very much is the perfect summary of the problem we're solving. So, how did we come up actually with this problem? It's also a great explanation for why venture studio make sense. So, we were actually working with some of the car manufacturers on various topics around machine learning and realized that they had a common problem. There was so much data coming in and it was just lying in some buckets, sometimes in the cloud, sometimes on prem. Firstly, they didn't really know well what to keep and what to throw away, but also then even when it was about improving the algorithms or even testing the algorithms of their automated driving features, there was no neat way to access the data. This was in cases where there were just a few dozen terabytes. And once your fleet, as you mentioned, for Tesla, goes up in the millions, you drown in petabytes of data and you need to really prioritize and understand this data. So, we've built the Siasearch platform where you basically throw in all the raw data without any labels. It could just be from your cameras, it could be from LIDAR, but also other sensors, also map data, data about the car. So, what Siasearch does is basically analyze all this data, trying to also infer context, like what's really happening in the scene, like is this like a busy street, is it not a busy street, are there any pedestrians doing something, are there any cars around, what kind of street is it, what's the weather conditions. So, it adds a ton of metadata, actually hundreds of things, that's one part to the product. And then the other part is, it makes us all searchable. So, you can very easily say, "Hey, give me all the data where ..." I don't know, like a pedestrian was crossing very close to the car or where, for example, you've had a cut in on some highway, "Give me a thousand of these scenes." and then basically, our querying engine pulls all this data very quickly and then you can use it for your use case. So, we believe this is very important, first, for using this data, but then also to reduce storage because in the end, you don't want to store hundreds of petabytes or exabytes of data, because right now, they're just using 5%. They might still only use 5% in the future, but it's important to use the right 5%, because all the data about empty highway drives or so, they're not so relevant. The algorithm learns pretty quickly how to drive on a highway. So, that is data maybe we don't want to store that much about but if there's some super interesting critical situation interacting with pedestrians, you probably want to store all these sequences. So, the software also helps you to prioritize that. You basically just put it on top of your cloud environment, whether it's one of the big cloud providers or some on-premise cloud, and then it helps making sense of that data.
Jon Krohn 37:40
Nice. I like so many of the business ideas you've mentioned on today's show, I absolutely love it. And this was the context that you initially came up on our podcast last week because we were talking about companies that exist out there to add structure and labels, metadata, like you mentioned, to images. There is a huge amount of opportunity in the last few years, I think, for companies to be doing this kind of thing, and it sounds like you guys have really found a great niche here. From what I understand, there's a lot of different car companies that are working with your technology. Maybe you can't mention them specifically, but I think I'm right about that, right?
Rasmus Rothe 38:20
Yeah, exactly. And it's interesting for anyone in this tech, for the large OEM brands, the suppliers who produce sensors, the autonomous driving companies, basically everybody in the whole stack of automated mobility. It could also be companies producing robots in the context of logistics, even drones. I mean, we haven't done stuff in that space yet, but I think you could even expand that to other use cases where basically, you have mobile devices producing a ton of data.
Jon Krohn 38:48
Yeah, that makes perfect sense to me. Did you know Alex Flint, who was also doing his Ph. D. when I was at Oxford?
Rasmus Rothe 38:54
His name rings a bell actually, yeah.
Jon Krohn 38:56
Yeah, he was in Oxford Entrepreneurs. I can't remember if you guys overlapped,if you were on the committee at the same time. He has started, a few years ago, a VC backed company in San Francisco, which specifically worked on autonomous driving for delivery. So, kind of like this drone idea that you just mentioned. We've seen things like Amazon mentioning that they would have technologies like this, where you have this four wheeled device that rides around on the sidewalk and can make last mile deliveries. And his company thought that this might be a relatively simple problem that would be easier than say, self-driving cars, but they realized over time, that it was actually exactly as complicated as making a self-driving car because there's pedestrians on the sidewalk. So, they ended up being acquired by General Motors. General Motors has a big autonomous driving division and that ended up buying up Alex's company. He's off working on similar kinds of problems as well. So, interesting that that's going on. All right. Well, to wrap things up here, fill us in on your vision, Rasmus, for the Merantix AI venture studio or your career. You've had an amazing journey so far. What's next?
Rasmus Rothe 40:23
I think the biggest thing on our agenda is now building the next batch of companies. So, I think you mentioned that earlier just before Christmas and we announced it in January, we raised another 25 million euros, which we will basically use to build around eight new companies in the next four years. So, right now it's really all about identifying those topics, assembling those teams, and also figuring out what kind of companies we will build. So, that's one big part. The other part is kind of building even more synergies between our existing and new companies and growing this Merantix ecosystem because I really believe that having a lot of companies–AI companies–next to each other in close collaboration, in a very trusted environment, will just help them all to be more successful, sowe want to institutionalize this a bit more. Then lastly, Merantix is kind of my 95% or 98% job. The other couple of percent I actually spend with the German association of AI companies, which I co-founded two years ago. We basically are around 250 companies now and it's kind of the intersection of, I guess, startups and small-medium businesses that use AI or are AI first and working closely with the German but also European government and being in an exchange to see what needs to change in terms of regulation or if there's no money that is being invested in the space of AI from the government side, what are efficient ways to allocate that. So, kind of building the bridge between the policy side and the startup AI world is another thing I spent some time on and I'm very excited, because I believe we're still only in the beginning. You could say that the early AI hype is a bit over now, but I think now the next ten, fifteen, twenty years, will be super exciting when it's about actually seeing all these things in application in the real world, like in radiology, like autonomous driving, and all these things. So, driving this forward, yeah, I'm very excited about it.
Jon Krohn 42:35
That is awesome. I had no idea you were doing that. It's such a great complement to the other work that you're doing, to at one end be growing startups quite pragmatically and saying, "Okay. Where are their business opportunities for us to be applying specific breakthroughs in machine learning?" and then on the other end, you're saying, "Okay,how are we going to make sure that society is ready for these changes and that politicians are working with us to come up with appropriate regulations?" This is so important because we need to make sure that the progress that we're making in AI isn't all of a sudden cut short by some societal or political backlash. I think that there is definitely a risk of things like that happening. I think that sometimes big tech companies can be using people's data in a way that makes users uncomfortable. So, we end up with regulations like GDPR, which I think are great. I think that it's important to be looking after citizens’ data and citizens should probably have more control over their data. But we also want to make sure that simultaneously, there's a balance that means that we're not putting a chokehold on machine learning advances. So, brilliant. Finally, Rasmus, as should be clear to listeners at this point in the program, you've attained a tremendous amount of success in your career for anyone, forget for someone as young as you only a few years out of your Ph. D. Do you have one particularly valuable nugget of advice for listeners who'd like to scale machine learning applications commercially themselves?
Rasmus Rothe 44:23
One thing I'm very excited about is self-service solutions in the context of machine learning. Obviously, there are applications of vertical AI solutions, like what we do with Vara, where we basically solve one specific problem with machine learning very well but you also end up having a lot of problems out there, which are very specific to the company. So, it might be some small manufacturing business that maybe wants to use deep learning for some visual quality control and it's something which is so specific to what they do that there will probably never be a product out there. But at the same time, what's out there on the open source side is still, yeah, you can download some open source code, but then you still need to host it somewhere, you need to make sure it's robust, you need to deploy it. And so, I think there's a big opportunity in the next couple of years to build kind of self-service products where small companies, but potentially also large companies, can basically customize some machine learning workflow, whether it's computer vision or natural language processing. So, I think that will be a very exciting space to start a company in the next couple of years.
Jon Krohn 45:35
That sounds great. Thank you for that, Rasmus. So, it's been a really interesting program. We've been able to cover so many different news topics through discussion with you directly, which is a kind of format we haven't had on the program before, but I think works really well. So, we were able to talk about scaling AI businesses, we were able to talk about applications to radiology specifically, and we were able to talk about autonomous driving news as well. So, that covers our content and today's A4N program. Rasmus, is there anything coming up soon that you'd like our audience members to know about, or would you like to provide them with details for reaching out to you, a Twitter handle, or anything like that?
Rasmus Rothe 46:23
Yeah, feel free to reach out to me on LinkedIn, Rasmus Rothe, or my Twitter handle, @RasmusRothe also. So, yeah. Always, always happy to start conversations around machine learning, around applications, around investment in the space. There's something very exciting we are planning right now in Berlin, which will basically broaden the Merantix ecosystem. I can't tell you much about it yet, but we'll probably announce it in July. So, follow us and then you will see what's going to happen.
Jon Krohn 46:54
Nice! Very exciting, Rasmus. I can't wait to find out what that is. Listener, thank you very much for your time and attention today. We greatly appreciate it. Please consider liking, subscribing, or following the A4N Podcast. It's available on Apple Podcasts, Spotify, Google Podcasts, and YouTube. And yes, feel free to reach out to me as well. Like Rasmus, I'm on LinkedIn, and in terms of social media, as with many people in this space, that is my most active social media channel. Of course, all of our handles, any links that we mentioned in today's program, I will be including in the blog post related to this podcast episode on my website, JonKrohn.com. I do have on my website, there's an email newsletter on the homepage that I recommend that you sign up for. That is the best way to stay in touch. Any new podcast episodes, lectures, free videos that I make, I have tons of free content that I create and I share via that newsletter. So, I do recommend that. I am also on Twitter @JonKrohnLearns. Thank you very much to our producer, Sangbin Lee, for production and editing, and of course, our guest, Rasmus. Thank you so much for being on the program. We will catch you again here soon.
Rasmus Rothe 48:17
Thanks, Jon and Sangbin. It was really fun.