GRTiQ Podcast: 98 Marc-Andre Dumas

The following podcast is for informational purposes only. The contents of this podcast do not constitute tax, legal, or investment advice. Take responsibility for your own decisions, consult with the proper professionals and do your own research.

Marc-Andre Dumas (00:00:19):

So I listened to a presentation about a Graph in Denver and I forgot about it for a while. And fast-forward summer 2020 and I wake up and realize how stupid I was to overlook the usefulness of subgraphs for building tech front end. At that point, everyone was using The Graph. So yeah, I started paying a little more attention.

Nick (00:01:11):

Welcome to the GRTiQ Podcast and the first episode of 2023. Today I’m speaking with Marc-Andre Dumas, an Indexer at The Graph. Marc-Andre launched his Indexer operation, Ellipfra, during Mission Control, which means Marc-Andre is one of the OGs within the Indexer community. Since that time, Marc-Andre has become a very active Indexer within The Graph community. And as you’ll hear, he’s now participating in the MIPS program.

(00:01:40):

During this interview, Marc-Andre talks about his career before entering crypto and his journey into web3.0 and becoming an Indexer at The Graph. Marc-Andre also shares important insights on the MIPS program, zero-knowledge proofs, Firehose, Substreams, and his vision for the future of crypto and The Graph. As always, I started the conversation by asking Marc-Andre about his educational background.

Marc-Andre Dumas (00:02:05):

Well, from a young age, I was interested in electronics and computers, so I naturally went to study computer and electrical engineering in college. Then I have work in various industries doing integration and system engineering. What’s system engineering, you might ask? Well, basically that means I was designing systems that meet customer specific needs. For example, I once worked for a broadcast equipment manufacturer.

(00:02:32):

Customers would come to us looking for hardware products for a specific use case. So in that context, think HD signal conversion or transmission of video over a distance. And we would have to figure out how to assemble a system using equipment we had in our catalog because the goal was to sell more equipment, right? So sometimes salespeople would promise features that didn’t exist and it was my job to design the overall system, work with the development teams to expedite the future development and sometimes even write code myself to make everything work together.

(00:03:09):

So I did this work in the broadcast industry and the transportation industry, working on trains, subway cars. Eventually, I returned to the broadcast industry and focused on engineering and operating large scale streaming video system. So that’s where my large scale engineering and met more standard IT, computing, technical operations. This work helped me develop tech op skills and gain expertise I’m high performance server and storage system. This is crucial when millions of people are trying to watch different things at the same time. The video has to load instantly with no buffering and at that specific job it was very important that there was no degradation of quality whatsoever. So that requires some very advanced engineering work.

Nick (00:03:59):

So Marc-Andre, what is it about system engineering and technology that interests you as a field? Why are you passionate and have pursued that in your life?

Marc-Andre Dumas (00:04:08):

Well, it’s all about building stuff. So you can build small things, you can write code, you can design a software system. You can try to build large scale things like you can try to build a bridge or something. And for me, I guess it’s somewhere in the middle. Try to assemble a few systems, a few softwares, some pieces of software and try to build something else completely. It’s kind of like building a Lego set.

(00:04:32):

You have specific blocks, specific bricks. They do certain things and you can build them in a standard way, but you can try to modify them. You can build something else differently. You can take your Lego blocks, Lego bricks and use, I don’t know, a key and ax or another construction system and try to put everything together to do something completely original. So as an engineer we tend to specialize.

(00:04:55):

You have software engineering, mechanical engineering, electrical engineering, but when you’re doing system engineering, you’re touching all these things together. You’re not an expert in anything, but you have to have basic understanding of all these different fields so you can use the different technologies and the different systems to build something different. Building stuff is what it’s all about.

Nick (00:05:18):

You’re joining me from Montreal and I’ve had a few guests of the podcast that have joined me from Canada and other parts of the world. What can you share with listeners about the crypto web3.0 scene where you live?

Marc-Andre Dumas (00:05:30):

Well, Montreal’s crypto scene is really cool because it’s mostly made up of independent researchers and developers who are working on all kinds of project. I would say that every big DeFi or blockchain project has at least one developer in Montreal. You will know because there aren’t really regular meetups or major conferences in the city. But I met a lot of these folks on flights to and from conferences. So that’s always very funny.

(00:06:01):

There are some legit crypto companies though. Obviously I have to talk about streaming fast, one of the core teams at The Graph, they’re based in Montreal. They’re working on scaling the protocol with their Firehose and substreams products. There’s also Zapper, a popular DeFi dashboard and tool provider that’s headquartered in Montreal. Another interesting thing is that Montreal has one of the few permanent NFT art galleries in the world. It’s called Zero X Society.

(00:06:31):

And despite maybe the downturn in the NFT markets in the past couple months, they’re still very active with regular exhibitions and organizing events. So, the city is known for creativity and innovation and that’s transpiring in the place crypto as in the crypto space.

Nick (00:06:49):

Do you have a sense for why you see so many different developers and projects springing out of there?

Marc-Andre Dumas (00:06:55):

Well, Montreal has several major universities. There’s a lot of research. I know some universities have blockchain classes. There are some research jobs in the space, so that’s probably explaining why many people get out of college, they were exposed to it while studying and now they want to get into crypto. And there’s some side research, some side researchers in cryptography itself, distributed systems, and that’s probably why it’s spurred so many different independent developers even though there’s no large, well, very few large companies in this space.

Nick (00:07:31):

So Marc-Andre, let’s go back in time then. You’re working as a systems engineer, you’re doing really cool work at large scale solving really important problems in an exciting space, broadcasting. But at some point you became aware of crypto. I’m curious when that was and what those initial impressions were.

Marc-Andre Dumas (00:07:48):

Well I cut the crypto bug back in 2016 a bit by accident, I guess. A friend showed me an infographic about Ethereum and its potential uses. And it really grabbed my attention. The infographic was all wrong. There was nothing I accurate about it and that’s only something I understood sometime later. But it forced me to revisit the concept of blockchains.

(00:08:15):

I had heard about it before but didn’t really think much of it. I think money wasn’t say, something that was interesting me and money in the sense of Bitcoin as a new alternative to money. But then when I started researching it, I was blown away because the way Ethereum was using the blockchain concept was basically making unsolvable technical problems solvable. As an engineer, that was a shock to me. And there was all these problems that I thought were unsolvable and suddenly, well, maybe we could design, we could use Ethereum to design something with a workable solution to these problems.

(00:08:59):

So I was especially excited about the potential for transparent and verifiable elections. Also started thinking about how it could help solving some sort of subtypes of tax evasion. And the idea of a world wild computer that anyone could access at the same time is still blowing my mind. And back then I knew it was going to be huge because of all the possibilities it was opening.

(00:09:32):

To me, it was an aha moment, like similar to when the internet started becoming popular or when Apple launched the iPhone. I had no idea where that could lead us, but I knew it would be big. So I spent a lot of time observing what was happening in crypto in the crypto world and trying to figure out where I could contribute. There were a handful of project developing on Ethereum, so it was fairly easy to track them down. There were so few of them.

(00:10:01):

It was really exciting to see all the big ideas people had and it felt like a gold rush to find new use cases for the technology. And for a while I would, I’d just watch and learn and eventually the writer [inaudible 00:10:17] came along and I joined the integration team at MakerDAO. I worked there for over three years and it was a great experience. I got to be part of a project that was building the foundation of what would eventually become DeFi. It was a very fun ride.

Nick (00:10:34):

When you think back to your engineering experience before you made the full-time move into web3.0 and blockchain, did you use a lot of that experience and skill immediately in web3.0 or did you have to relearn some things?

Marc-Andre Dumas (00:10:48):

Well, obviously there were new concepts I had to get familiar with, but many of the basic system engineering integration, let’s say good practices were also applicable to a different field like crypto and blockchain and still is. And I still use this knowledge and experience today, and that’s what brought me to MakerDAO. At the time they were growing their integration team and integration system engineering are related to each other. So I was able to use that knowledge of basic good practices of system engineering into crypto, especially at MakerDAO at the time.

Nick (00:11:26):

I’m curious about that system engineering lens, this system or framework by which you earned a living. It seems to me that it’s a process whereby you’re optimizing different things to address the solution. And I’m just curious, Marc-Andre, do you think that type of lens, that type of framework would be helpful to people who want to improve themselves, a self-improvement framework?

Marc-Andre Dumas (00:11:51):

Well, I think that there’s some people that are applying that. I mean, there’s the old school of thought that measuring everything they’re doing. I mean with smart watches, that’s slowly something that gets more data about your life. And you could count calories or time exercise and everything. And maybe I guess that could be something useful for some people, but I would not necessarily make it a general philosophy of life because there are much more to life to just data or performance.

(00:12:23):

I mean, think about heart, music, personal relationship. That’s not something you can necessarily measure. I don’t think we should be measuring the quality of a personal relationship. So, we need to be able to let go. We need to be able to go with the flow. Quantifying everything is probably not a good approach, I would say in life in general. If you want a fulfilling life, yeah, don’t try to measure it, just live it.

Nick (00:14:27):

Marc-Andre, how would you explain to somebody who’s familiar with web2.0, familiar with the internet, how it’s different or why it’s better to build in web3.0 or build on Ethereum? Because here you are, you’re a systems engineer, you understand technology very well, you solve problems at a large scale for a living, and yet something about building on Ethereum, moving to MakerDAO was more appealing, more exciting to you. How would you explain why that’s the case?

Marc-Andre Dumas (00:14:54):

Well, to me, what was appealing, it was the new space. It’s like the far west. Everything had to be built, everything was new. So someone who’s looking for challenges, they’re trying to a place where they’ll have the opportunity to leave their mark, right now, blockchain web3.0 Is the place because we’re trying to solve all these engineering problems that are completely new and they’re very different than what we would see in Web1.0 or web2.0 or anything, any other industry.

(00:15:27):

So we have to come up with good solution to these problems. And what’s fantastic about it is that if you find a really good solution, you’ll see that a couple of months later or a couple of years later, someone else will pick up exactly the same solution. So that proves that they left their mark industry and you design a good solution that is applicable to multiple problems. So for one like me, that’s super exciting.

Nick (00:15:50):

When you think about the future and this interplay between web2.0 And web3.0, sometimes in past interviews I’ve asked guests, does web3.0 win and web2.0 goes away or is there an environment where they both kind of coexist and serve different users? How do you think about that future and how web2.0 And web3.0 coexist?

Marc-Andre Dumas (00:16:10):

Well, it’s hard to say how this will coexist in the future because I don’t think anyone has a clear picture of what web3.0 will become. We’re still very, very early in the field and there are many, many things that web3.0 is maybe not so great at doing right now. Hopefully it’s only temporary and eventually we’ll have good solutions and web3.0 will be so much better than web2.0 And everyone will move there. But until we’re certain that web3.0 will be 10 times better than web2.0, I would say that there will be coexistence, but that’s not the ambition.

(00:16:52):

The ambition is that we’ll have something 10 time or 100 time better than web2.0 Or everything else we had before. So this transition will be obvious and they will want to transition easily and there will be just a new web. That’s it.

Nick (00:17:07):

When you think back to that time where you decided to go full-time into MakerDAO and work on the integration team, what were your thoughts at the time? Were you nervous about jumping into a new industry whose future is unknown? Were you framing it as a big career risk or career change, or did you feel just so much conviction and confidence that it was a very easy decision and you jumped at the opportunity?

Marc-Andre Dumas (00:17:30):

Well, obviously there was a career risk doing that move back in 2019. Yeah, there was a career risk. Going back would be difficult probably, but I was very confident and was already spending a lot of time, nights and weekends exploring the space. So I knew there was something to do there and there would be a lot of potential. But it took me a while to really believe that I had this opportunity. I was so happy when I was given this opportunity and until the very last minute I couldn’t really believe it.

(00:18:03):

But, eventually I quit my job and I went full-time to crypto and today there’s basically no way I will ever go back to something else. I mean the space is so exciting, it’s so rewarding. Everything you’re doing is so exciting, I can’t go back to a irregular engineering job.

Nick (00:18:22):

So in 2019 you make the move to MakerDAO, you go full-time into the space and eventually you become aware of The Graph because we’re talking today, Marc-Andre, because we’ve met in The Graph community, you run an Indexer at The Graph. When did you first become aware of The Graph and do you recall what those first impressions were?

Marc-Andre Dumas (00:18:39):

Yeah, well it was back in early 2019, I think even before I joined MakerDAO. And I must confess, The Graph didn’t make a great first impression on me. As part of my exploration of crypto and the crypto and blockchain space, I had been trying to track every single project in the space and it was getting overwhelming. There were so many new projects popping up and most of them weren’t very credible or were just rehashes of old ideas. We had just experienced the ICO craze. So that’s the context.

(00:19:17):

So I started using simple mental filters to quickly determine which project were worth paying attention to. And as part of these filters, I had the filter saying that, well, if a project has a well funded marketing campaign, that’s a big red flag. So The Graph was a major sponsor of the E-Denver that year, and I had never heard of them before, so it didn’t start off with the right foot on me.

(00:19:48):

I mean, it was a quick filter, it had a good hit rate. But yeah, it failed sometimes and it failed with The Graph. I was also building a time analytics reports for MakerDAO, and so I understood the difficulties of extracting blockchain data, but also from that perspective, I didn’t see a huge market for analytics and I thought it was a small niche that I already had a good set of players. So I listened to a presentation about The Graph in Denver and I forgot about it for a while and fast-forward summer 2020 and I wake up and realize that how stupid I was to overlook the usefulness of subgraphs for building tech front end. At that point everyone was using The Graph. So, I started paying a little more attention at that point.

Nick (00:20:36):

Can you explain that aha moment for you as it pertains to subgraphs, because I think that’s something that’s often overlooked, but in my opinion, The Graph has set the standard for querying and indexing blockchain. Data and the way they did it is by introducing subgraphs and then subgraphs being so widely adopted. So how would you explain for listeners that don’t quite understand the utility of subgraphs and what that enables?

Marc-Andre Dumas (00:21:02):

So blockchain is basically a write only database, so a database that is optimized to write data. So reading data from a blockchain is actually very difficult. You have it’s costly and processing power and the data that you get out needs some additional processing to be useful. So it’s not easy, it’s not simple.

(00:21:25):

It’s really optimized for writing, not for reading. So when you visit the website Adapt or a DeFi or web pre site, you’re actually reading data from the blockchain. So The Graph fits in between, in between the blockchain to raw data and the blockchain and the UI. And using subgraphs, it’s able to extract and make the data very quickly and easily accessible on the front end.

(00:21:52):

So it makes everything much, much easier for developers. They don’t have to focus on writing complex software to extract the data. They just need to write basically some specifications about the data they want to see. It’s made very easy using some libraries they can very easily display on their website or whatever application they they’re using. So it was a great time saver for developers and they don’t need to worry about running servers, paying huge AWS or Google Cloud bills every month. It’s all end all very easily by The Graph.

Nick (00:22:27):

So you have this moment where you realize, okay, I was wrong on my heuristic that The Graph was a mirage. There’s something to what they’re doing here. What then is your pathway into The Graph and what you become today?

Marc-Andre Dumas (00:22:42):

Well I think around the same time I was already running a personal Ethereum archive node. As I said, I was doing analytics and stuff with that. And it was very rare for someone to do that, was tough and required a lot of storage and there were no real reason to do that except being, I don’t know, curious and someone who liked to tinker with servers and stuff.

(00:23:06):

But the difficulty of running an archive node was becoming a problem. So people in the Ethereum community were trying to figure out a way to at least make the data from archive nodes more accessible. And that’s part of one of these discussions. Someone from The Graph, and I don’t remember who said, well hey, we have a solution for this. Anyone running an Ethereum node can join our testnet and they’ll get incentivized to serve data.

(00:23:36):

Well, at the time I had an archive node, I had some spare time and I was curious about The Graph. So thought to myself, why not? So I’ll give it a try. I signed up, got into the Mission Control program at the time, and I spun up an Indexer. It was a fun little experiment. It was interesting to see how The Graph worked and especially in the context of making Ethereum data more easier to access to anyone.

Nick (00:24:05):

For listeners that aren’t familiar with the Ethereum archive node, and I have to admit, I myself don’t have a ton of background or knowledge about this, but doesn’t EIP4444, the four fours EIP, which is a hard thing to say, believe it or not, doesn’t that make some decisions within the Ethereum community related to archive nodes and people that might be running those? Have I got that right?

Marc-Andre Dumas (00:24:31):

Yes. So EIP 4444 is about eventually starting to prune data or removing data from the Ethereum nodes because right now Ethereum, like every blockchain is a write only database. So we keep adding data to it and it just grows. It never shrinks and that’s okay, that’s manageable. But if you picture long term, very long term, Ethereum will still be around 10, 20, 50 years from now.

(00:25:02):

Does it make sense to design a system that will grow independently for an infinite amount of time? Well, not necessarily. So now Ethereum community is starting to think about eventually make some of the old data inaccessible, at least for the standard node operator or using a standard node. But obviously there’s all data that is useful and it’s important for some applications to keep. And I think it’s generally good practice to keep all the archive at least accessible.

(00:25:35):

So I think the community right now is looking for alternatives. Again, The Graph came up and said, well, we have a solution for this also, and we might be able to provide a good way to make this data accessible to everyone for over a long term. So I haven’t seen the recent details about it and there’s probably a lot of missing pieces in The Graft protocol right now to be able to do that, but it looks like that’s going to be happening eventually. And that’s great that again, The Graph will be able to help the Ethereum community to grow and to scale.

Nick (00:26:12):

When you made that move to become an Indexer and join the first testnet, which was Mission Control, how did you frame that decision? I mean, did you see it as a hobbyist just trying something new? Or in your mind, did you think I might be making another career change here?

Marc-Andre Dumas (00:26:28):

Oh, not at all, right? It’s really I want to experiment, so there’s something cool. The Graph at the time, I thought The Graph looked cool, so I wanted to learn more about it. So it was perfect opportunity to experiment and learn about the details of The Graph.

(00:26:42):

So I had what was needed to run that. So I did that and just, it was mostly for my own personal interest. The testnet was incentivized, but calculations I made at the time it was going to be roughly covering my costs. So, to me it wasn’t necessarily something interesting by itself. I saw it more as a learning experience.

Nick (00:27:06):

So Marc-Andre, you joined the network then as an Indexer participating in Mission Control. And again, for listeners who don’t know, Mission Control was the very first incentivized program and The Graph recently released a second program called MIPS that we’ll talk about in a minute. But Mission Control for me is really an interesting thing because Marc-Andre in introduced the first generation of Indexers, many of whom like yourself are still around. What would you share with listeners that weren’t around for that period of time Mission Control in that first generation of Indexers? What was that experience like in Mission Control?

Marc-Andre Dumas (00:27:38):

Well, it was very, very fun. I have very good memories of Mission Control. It was a great experience. It was a great learning experience. I didn’t know much about The Graph, but also realized I was lacking a lot of knowledge and the many technical operation tools like Kubernetes. To me, that’s something I didn’t know. And for those who don’t know, that’s an easy way to scale large applications on the cloud.

(00:28:03):

So I had a lot to learn. I made a lot of mistakes. I end up having to rebuild my whole setup a few times. And then to test our system, the team would send us a huge plot of queries to our system, to our Indexers, and that would inevitably crash my servers. So I had to come up with solutions and architecture to be able to withstand these demand peaks. It was really fun to me as a system engineer with my background. That’s exactly the type of challenge I like to work with and I like to try to come up with solutions for tough problems like that.

Nick (00:28:44):

How long would you say it took for you to get comfortable with receiving those tests and that flurry of queries to your servers? How long until you felt like, oh, this isn’t going to crash my system anymore, I’m now at a point where I can do this?

Marc-Andre Dumas (00:28:59):

Well, I had to go back to the drawing board a few times. I think maybe the third iteration. So things were relatively quick back then. So I think maybe after three months of playing, eventually I got something that I think was good enough. And very shortly after Mission Control we moved to mainnet. So basically the design that was working for Mission Control became the production Indexer for mainnet and it was running fine and basically the same design is still in operation today.

Nick (00:29:30):

You kind of answered this there, but I just want to double click on it. Is that move for an Indexer from a test net to mainnet, is it a difficult, arduous process? You said the setup was kind of working in test net, but the migration of it, is it difficult?

Marc-Andre Dumas (00:29:48):

It was really simple because at least the way I built my test net set up, it was very easy to simply copy, paste the setup to do something working for mainnet. So it was really simple, but today in general, if you want to move to mainnet as an Indexer, there’s a economic challenge because you need to stake a certain amount of GRT to be able to run an Indexer. I guess that’s a difficulty for most people to do that. I was lucky as part of the Mission Control and as part of the transition to mainnet, I was basically been given the amount of GRT necessary to kick off the Indexer. So I was very, very lucky about that. So that make it very easy for me.

Nick (00:30:33):

We’re now in a new program, which is announced back in August or September of 2022, which is MIPS. And MIPS is now adding new chains to The Graph Network. Started with Gnosis. And then at the time of this recording just announced Polygon, you’re participating in MIPS, you’re back in a different program here. What’s that experience like and what’s different?

Marc-Andre Dumas (00:30:56):

Well, it’s very exciting because now I have this set up and I have to basically grow everything. I had to add multiple nodes because I used to have just single Ethereum archive node, but now I need archive nodes for Gnosis, Polygon, there will be many more. There’s some interesting engineering challenges there.

(00:31:19):

So again, great challenges to try to solve and also try to solve it in a way where it’s going to work, it’s going to scale. Because right now we’re talking about maybe 10 chains, but eventually, I don’t know, it’ll be 50, 100. I don’t know if this thing will keep growing. So we need to figure out a good solution on the long term.

Nick (00:31:38):

Explain to me the rationale that you had when presented with participating in MIPS and adding these new chains. Because it could be argued that you found your swim lane with Ethereum chains and you are doing fine, and then MIPS comes along and you have to make this decision of, well, do I want to start serving queries on Gnosis and Polygon and then all the changes that might be added. What’s the rationale for you to say, yeah, I want to do that, I want to take on that?

Marc-Andre Dumas (00:32:05):

Well, I mean the first reason is for an Indexer to be able to maintain its position in The Graph Network, basically need to try to support as many subgraphs as possible. And at some point to support as many subgraphs as possible, you need to support as many chains as possible. So I run the Ellipfra Indexer. The sub Indexer is running pretty well. It’s relatively competitive.

(00:32:32):

And if I want this Indexer to remain competitive, I have no choice but to support multiple chains. But you’re asking a good question here because yeah, you can see that I’m mostly an Ethereum guy. I was brought into the space with Ethereum and I’m always very focused on everything related to Ethereum. So why deploy more chains?

(00:32:52):

Well, I think there are different motivations, but for me the work we’re doing integrating multiple chains will be useful for Ethereum itself because pretty soon, and I hope it’s going to be one of the next network in the MIPS program, we’ll start integrating Ethereum L2s. And when you would integrate a L2 versus say side chain like Polygon or Gnosis, well, it’s basically the same work. So the work we’re doing with these other chains are going to be useful for integrating L2s and very short term, I hope.

Nick (00:33:29):

Well, let’s talk a little bit more about Ellipfra. And forgive me, Marc-Andre, I can’t speak French and I can’t even get the accent, but Ellipfra is the name of your Indexer. What does that name mean? First of all, let’s talk about the name. Where did you get that?

Marc-Andre Dumas (00:33:44):

Oh, well, it’s very simple. I just took two different words that are related to each other in the space. So the first word is elliptic. So it refers to elliptic curves, which is a basic, a lot of cryptography that is used in blockchain words today. And the fra, that’s for infrastructure. So basically elliptic infrastructure or infrastructure to support elliptic systems. So that’s where this name comes from.

Nick (00:34:13):

And you’re presently a full-time Indexer, so we track your story here. You went from full-time at MakerDAO dabbling in The Graph to leaving MakerDAO and going full-time in The Graph. Have I got that correct?

Marc-Andre Dumas (00:34:25):

Yeah, exactly. So until last summer, I was still working on MakerDAO and I saw that the MIPS program was coming up, so I was facing a choice. I knew that to keep Ellipfra for competitive, I had to invest a lot of time to integrate all these new chains. That would’ve been incompatible with keeping a full-time job.

(00:34:49):

And I also saw a broader opportunity for independent operators in other protocols and services. There’ve been many things happening the in crypto space in 2022, but I think in a lot of cases it made the crypto community realize we are relying too much on centralized services. So now the community is reconsidering choices that might have been acceptable to bootstrap web3.o and DeFi, but now it’s time to be more serious about decentralization.

(00:35:22):

And my thesis is that there will be room for independent operators outside of The Graph, and Ellipfra will also be active elsewhere. So the timing was right for me to go full-time. And I’m not just going full-time. My plan is that I won’t be alone for long. Ellipfra will become a small team to grow things further. But again, as a system engineer, I try to get rid of single point of failures everywhere. And right now I am a single point of failure for Ellipfra, so I need to fix this. So there will be a solution coming up for that.

Nick (00:35:59):

Marc-Andre, one of the reasons we’re speaking today is you and I actually met face-to-face at East Denver, so you’re one of the handful of Indexers I’ve had the opportunity to meet in person. But you’re also, as you kind of referenced there and is well known throughout the community, Ellipfra is a high performing, very well operated Indexer. Without giving away maybe the secret sauce that you want to keep close to the vest here, how would you describe the ways in which you’ve optimized Ellipfra to perform as well as it does?

Marc-Andre Dumas (00:36:30):

Well, I think it’s a matter of spending time analyzing many different variables in the protocol and in the system. And I guess that goes back to my longtime experience in managing and building and operating large systems. You need to be able to pay attention to a lot of details and you need to be able to identify bottlenecks or things that could be optimized.

(00:36:55):

So over time, and it’s been almost two years now, I’ve been tweaking, adjusting, improving small little things here and there, and I’m keeping close attention to all of these data. So I have all of the strips, I have dashboards, I’m monitoring a lot of the data, and over time I identify pain points that need to be optimized and I improve them. And eventually over time, that makes the Indexer perform much better.

Nick (00:37:25):

So sounds like to me that, that background and systems engineering is part of the secret sauce here, this multifaceted approach to problem solving.

Marc-Andre Dumas (00:37:34):

Yeah, probably that’s the experience and the background I bring here. I know there are Indexers that have different backgrounds. Some would come with very strong experience in database and administration and I’m sure it helps them a lot. To me, that’s not a strong point for me, so I’m still learning about that. But we’re all bringing different experience. Some people are able to make their Indexer very performant on some specific aspect, again, thanks to everyone’s specific background and experience.

Nick (00:38:05):

Knowing what you know now, if you could go back in time and give yourself advice, and in some ways this could be advice to a listener who’s contemplating becoming an Indexer, but what do you wish you would’ve known then that you know now?

Marc-Andre Dumas (00:38:18):

Oh, yeah. So I completely underestimated how complex the task of spinning up an Indexer was. I spent a lot of nights and weekends working on it and I already had a full-time job in crypto, so time was limited. I also didn’t realize how important TechOps knowledge was for this task. As it was just not a question of running a single piece of software, it was running multiple different pieces of software together. And so that require a lot of TechOps knowledge for this task.

(00:38:51):

And to build a performant and reliable Indexer, you need a lot of experience and things like Docker, ideally Kubernetes, databases, administration, stuff like that. So you need to know how to run a Postgres database, troubleshoot network, latency, build fast storage. You can run an Indexer without that knowledge, but it is going to be maybe your next year will be have an average performance.

(00:39:17):

And in my case, I was lacking a lot of knowledge in all these specific areas and there was a steep learning curve. But again, I like to learn. That was a challenge for me. I learned about many different new technologies doing that, but others, and so anyone that comes in needs to be able to ready to learn and be ready to spend a lot of times to sharpen TechOp skills and also spend a lot of times to understand the intricacies of The Graph protocol.

Nick (00:39:48):

One thing that has always impressed me in interviewing or meeting different members of the Indexer community is the nature of that community. Because as someone looking from the outside in, it looks like a bucket full of competitors. But once you meet and speak with them and then see them in, for example, Indexer office hours, you also get a sense that they’re collaborating and working together. How would you describe the way the Indexer community works at The Graph?

Marc-Andre Dumas (00:40:15):

Well, I think you’ve get it right. There’s a very big spirit of collaboration and there’s a desire of help each other, especially among the independent Indexers. We’re all competitors, but we still manage to maintain cordial relations and we share our experience and knowledge with each other. And there’s also even a willingness to support new Indexers who join the network, especially when they show a desire to learn and they have some basic knowledge about operating blockchain services.

(00:40:48):

They’re always very well welcomed and the Indexers are always upbeat to help them. So I think the reason for that collaborative spirit in the community is that we all understand that building a successful decentralized service requires a diversity of players and contributors. Having more Indexers that increases the chance of success of The Graph. And if you increase the chance of success of The Graph, well that’s going to be ultimately going to benefit us all.

Nick (00:41:19):

I’ve had the opportunity to interview many Indexers from The Graph community, and in fact, I want to interview all of them. So for any Indexers that haven’t been on the podcast yet, reach out to me. I would love to speak with you. But Marc-Andre, if we throw all the Indexers into a bucket, we’re going to find an assortment. We’re going to find some really large, big teams.

(00:41:39):

We’re going to find some mid-size teams all the way down to individuals like yourself. These are sole operators, one man, one woman shop running an Indexer operation. What are some of the challenges that you’ve experienced as a sole operator? And again, I know as you said earlier, you have an ambition of growing your team, but presently as that sole operator, what are some of the challenges?

Marc-Andre Dumas (00:42:02):

Well, it just never stops. One of the challenge of being a single operator is the constant maintenance and adjustment that is required. There’s always something that needs to be tweaked or optimized, whatever, it’s at the server level, at the low level or in the subgraph allocations or the query cost model.

(00:42:22):

Automation can help to some extent, but there’s always some manual intervention that is required. And I mean it’s possible to operate with just very minimal operation. I know that some do just weekly or monthly tasks to maintain the operator, but that’s going to result in maybe subpar performance and maybe they’re not going to be serving a lot of queries. So to be able to compete in the Indexer market and ultimately provide a good quality of service to The Graph users, you need to put some constant attention and effort into it.

Nick (00:43:04):

How important do you think it is for The Graph as an Indexer community to be able to support and kind of onboard that single operator as opposed to only attracting large big teams? I mean, is it important that we keep adding or attracting future Marc-Andres to the network?

Marc-Andre Dumas (00:43:22):

Yeah, I think we need a path to allow newcomers to continue into The Graph ecosystem and eventually have them being able to make their place and being able to find their edge and be able to become an important player to The Graph. So we need to make sure there’s always a possible path for doing this. Can we build a network with only sole operators? Probably not.

(00:43:52):

So there needs to be a balance between larger, maybe more professional outfits and small individuals. But I think that the key here is really having this path that is possible. It should be possible for a dedicated knowledgeable person to be able to come in, learn how it works and build their place into the ecosystem.

Nick (00:44:20):

As an Indexer, if you use that systems engineer perspective,§ when you think about the role of Delegator and you think about the role of Curator, how do you think about how important they are as either partners or collaborators, if you will, in building The Graph ecosystem?

Marc-Andre Dumas (00:44:36):

I rely completely on Curators to tell me what is worth indexing and what isn’t, especially for independent operators, I don’t have a lot of time to spend to research the individual subgraphs and try to figure out which ones are worthwhile to index and which ones are not. So the curation signal is essential to me and I’m able to basically offload all this work to the Curators.

(00:45:03):

I think that’s a great thing in the protocol that we’re able to do that. But the curation signal isn’t perfect. But I hope that over time as the network matures, it will become better and better at identifying the worthwhile subgraphs. Maybe the curation mechanism itself will need to be upgraded or adjusted a little bit, but I’m not super knowledgeable there. But I’m confident that it will improve either organically or through changes to the mechanics themselves.

(00:45:35):

Delegators, that’s a bit different. And frankly, for a while I was considering delegation as more a purely as a crypto economic gizmo created to balance out the economics of the protocol. But now I see something I think more interesting and more valuable out of it. Delegators are providing a valuable feedback to Indexers and into the network overall. And when Delegator picks an Indexer, it basically tells them, well, as a GRT token holder, I think you are doing a good job for the network and please keep it up.

(00:46:15):

So this in turn makes the Indexer generate a little more income for itself. And over time, this can be valuable to maybe weed out Indexers who are not necessarily acting in the best interest of the network because you’re making the ones that are considering better, generating more income, and while the others will stay where they are. And over time, that could be improving the health of the network.

Nick (00:47:17):

How important is keeping stake decentralized at The Graph? It’s something that comes up on Twitter quite a bit, and there’s a few Indexers within The Graph community that talk a lot about it. But the idea here for listeners that aren’t familiar with what we’re talking about, the argument is that it is better for the health of the network to have many Indexers and stake with these Indexers distributed equally, as equally as possible, as opposed to having just one or two market leaders that capture all the delegations as a form of potential centralization despite the vision of creating a decentralized network. So going back to that original question, Marc-Andre, how important of a topic is this for you?

Marc-Andre Dumas (00:47:57):

Well, any decentralized network requires a variety of players and you need some diversity on people, players, or keepers in the network. So for most network, that means validators, miners, collaborators. You need as many of them to ensure that no single entity or cartel takes an effective control of the network. And in the case of The Graph, that’s certainly applicable, but there’s also an aspect in the quality of service that The Graph provides.

(00:48:32):

So it’s important that we have as many Indexers as possible because every time we are adding an Indexer to the network, we’re adding to the reliability and the quality of service. For instance, a new Indexer might be deployed to an underserved region of the world, maybe they could deploy to South America or South Africa or Southeast Asia, and which will allow users in these regions to benefit from lower latency in their request and basically make their application and webpage load faster.

(00:49:06):

So more Indexers means that we’ll have a better worthwhile coverage, but also increases the odds that we have a good coverage in the subgraph indexation itself. So because every Indexer picks a subsets of the subgraph, but together we will be able to index hopefully all of them. And that’s the quality of service aspect, but there’s also a security aspect. More Indexers means that we can effectively compare proof of indexation and query review results between the Indexers so we can ensure there’s no fraud going on and that Indexers are honest and they’re providing good and accurate data to the users.

(00:49:49):

So how do we ensure that we have a sustainable Indexer ecosystem? Well, we need to make sure that it’s possible for as many Indexers as possible to be profitable. So we can approach this problem from many different sides. We can ensure that the software is easy to use, we can provide automation tools to Indexers and so on and so forth. But at the end it boils down to query fees and the indexation rewards. And today this income is heavily dependent on the stake available to the Indexer.

(00:50:23):

So the larger the stake, the more income an Indexer is able to generate. And right now, if we look at the size of the stake available to Indexers, we can see that 50% of the allocatable stake is controlled by only eight Indexers. I have nothing against these eight Indexers. Some of them are doing a pretty good job, but we’d have a better network if that income would be more equally distributed. That would ensure a better diversity of Indexers.

(00:50:56):

What can we do about it? Pretty much the only way is to educate Delegators to make better choices for the interest of the network when they pick an Indexer, but honestly, we won’t get very far with that. Doing so requires a lot of knowledge about the network and many others decentralized network face the same type of centralization dynamics.

(00:51:18):

So there’s no silver bullet. One interesting thing that we could do, and we had a very recent example and that it’s interesting because it’s basically we’re gently influencing Delegators in a subtle way. There there’s been a recent change to the official grand delegation interface to sort the Indexer list by the amount of query fees that’ll be delegated instead of the size of the allocation.

(00:51:43):

And amount of query fees, I think it’s a better proxy for the performance of an Indexer compared to the size of the allocation. So basically when a Delegator wants to quickly delegate to an Indexer, well they’ll see on the front page a list of Indexers that are probably better aligned for the network. And I think what could be argued as better Indexers, I mean put in spotlight instead of just the bigger ones.

(00:52:15):

On that topic, Jim Cousins who ran the independent Indexer Wavefive recently made the commitment of not growing his Indexer stake beyond 3%. And that’s something very admirable, and I commend him for doing that. But is it the solution? I have? My doubts. Well never have a perfectly distributed stake with everyone being equal. That’s impossible. Well, is there a point where an Indexer gets too big? Yes, obviously, but I think that point is probably higher than only 3%.

Nick (00:52:53):

Do you think we’re doing enough in the community either at the Delegator level or at the macro level where all the stakeholders communicate in the forum or other places in addressing this and keeping it top of mind that we must come up with mechanisms or processes to keep stake decentralized?

Marc-Andre Dumas (00:53:10):

I think it’s been everyone’s mind since early on in the network. I think it’s okay. And as I said, there’s no silver bullets. It’s not like if there’s a easy and obvious solution that we could use that will solve the problem. I don’t think there’s any. I don’t think we’ll ever find one. So it’s good that we keep this in mind. We need to make sure that in the educational material and also in the dashboards, we present things in a way to make sure that Delegators understand these challenges.

(00:53:44):

But there’s no perfect solution yet. And I think generally speaking, we’ve been doing a good job. And if we look at the size or where these large delegation comes from, let’s say the top eight Indexers, it generally comes with large funds, large investment firms. Some of them is also coming from exchanges that are offering staking services.

(00:54:09):

So making these players revisit their decisions, that’s more like a business development question. Well, maybe Indexers, smaller Indexers hire the BD person than try to obtain these larger stake in these allocations. But we’re not talking about something that isn’t necessarily accessible for the smaller players or the newcomers. And I think that’s where our focus needs to be. We need this path for a small individual to be able to come in, being able to slowly grow the position into the network so it can become a good player in the ecosystem. So that’s where the focus should be.

Nick (00:54:47):

If you project out in your mind and you think five, 10, 15 years from now, how many Indexers do you think participate in The Graph ecosystem if it does what it needs to do?

Marc-Andre Dumas (00:54:58):

We have a couple of hundreds today, and that’s not a bad number. As I said, we need to make sure that running an Indexer is profitable. So there will be a natural limit in the number of Indexers we can have. Maybe we’ll start seeing some specialization from the Indexers. Maybe some will specialize in specific chains, maybe others will specialize in certain types of subgraphs. I don’t know. Well, maybe in this scenario, smaller Indexers with slim and simple operations will be able to remain profitable and then maybe will sell higher number of Indexer than, but let’s say in that general all-purpose Indexers, that’s always going to be relatively limited, I think.

Nick (00:55:47):

What are some things that excite you or you’re excited to see in the upcoming maybe years or months as an Indexer at The Graph?

Marc-Andre Dumas (00:55:54):

Well, as an Indexer, Firehose and substreams is very exciting. It should come first, and they’ve been coming first and for a while, so we’ll see. But it’s very interesting to me because it comes with a whole set of new challenges. Indexers will have to make major changes to their infra and they’ll have to reconsider the way that they operate to support us. And that’s exciting for someone like me. And maybe that’s not very well known or understood because Firehose and substreams are going to make things much, much better for most users of The Graph. But it’s creating challenges too. It’s a great positive, but it won’t make the Indexer job easier, far from it.

Nick (00:56:38):

So I imagine Marc-Andre’s a non-technical person that this is just kind of table stakes for being an Indexer, a new technology is introduced or new ways of doing things and there’s an adoption curve and you have to learn and take those on. But in the grand vision of things, am I correct in thinking about the nature of Firehose and substreams is that it’s a huge value add to users and really kind of a pivotal moment in The Graph ecosystem?

Marc-Andre Dumas (00:57:04):

It’s a great improvement in performance for everything. Things that are extremely difficult right now will become much easier and it will be much faster, and that’s great. But that performance will come from bigger, larger, and more complex infrastructure that the Indexers will need to deploy in the coming months and years.

Nick (00:57:26):

So similarly, Marc-Andre, if you project out your vision for The Graph and what it can do in the future, what’s your future vision and what makes you most excited about the future of The Graph?

Marc-Andre Dumas (00:57:38):

Without question, the future is verifiable queries. Right now, we have to trust Indexers to provide accurate and truthful data. And there are me mechanisms in place to slash Indexers that are providing inaccurate information. But these mechanisms are not strong enough for some potential use cases of The Graph. Using verifiable queries, we’ll have cryptographic proof that the answer provided by the Indexer is accurate and matches the on-chain data.

(00:58:12):

That’s a huge undertaking. It will probably take years to get this done, but once we’ll have this, it will open many use cases, especially for applications where we need to loop process data back into a blockchain. Oracles and cross chain bridges are example of these applications, and if it can be done properly and cheaply, The Graph will become a way to scale blockchains by allowing trustless off-chain data processing and execution.

(00:58:49):

So then The Graph will become an essential infrastructure to allow blockchains to scale. I mean, it’s usually ambitious, but there’s a path to get there and I really hope the different core teams and research teams are working on it right now. I really hope that they find the right way to do this.

Nick (00:59:12):

For listeners that want to track that idea or the progress of what you’re proposing there, the way to do that is what? Check in on the status of zero-knowledge proofs or snarks? Or what’s the technology they should be following?

Marc-Andre Dumas (00:59:24):

Yeah, it’s basically a zero-knowledge proof. So basically when an Indexer will provide an answer, it will also provide a small proof that could be executed and that will confirm that the data is true and accurate is true. And if listeners want to learn more about it, look for shellproof. That’s something that was announced back in The Graph day in June, 2022. That’s the vision. If they’re able to fulfill this vision, that’s going to be very applicable.

(00:59:56):

Imagine you’ll be able to use data from a blockchain, let’s say from Ethereum and using other data from a completely different blockchain, I don’t know, Fantom or BSC. And you’ll be able to take this data, merge it, process it, generate a proof that will combine data from two different chains. So once that’s possible, that means that you can use it to bridge assets from one blockchain to the other.

(01:00:23):

You can run maybe oracle price and I’m sure there will be other uses that no one has any idea of right now. But basically there’s, and I mean bridges and oracles are all things that are existing today, but most of them require to always have a trust element. But if The Graph is able to generate these proofs and allow this processing to be done completely trustlessly, that’s going to be opening a lot of a new dimension and new ways to scale everything

Nick (01:00:56):

Going back in time. You said when saw Ethereum, you kind of used that systems engineer and that background you had to realize that Ethereum would unlock a lot of potential in the world, that it was a solution to a lot of things that people could really use. I’m curious if we apply that same type of experience and frame of systems engineer to The Graph, do you also see it as kind of like that solution to a problem that exists in web3.0 that it is designed or engineered to address?

Marc-Andre Dumas (01:01:30):

Well, with verifiable queries, if we’re able to solve that, we’ll have a great social problem. And the social problem is, well, how can you tell if something is true or not? I mean, that’s a basic problem everyone had for a long time, right? But now with verifiable query, we can tell, well, that’s accurate. That’s the truth and the truth as defined on blockchain, right? So maybe it’s a limited truth, but it’ll be proven to be exact and truthful. You solve that, that’s going to be making a lot of things much, much easier. And if you do things much easier, allowed to think much easier, you are also opening new use cases and new applications. So I’m really looking forward to this.

Nick (01:02:18):

Marc-Andre, in 2019, you made an incredible career change joining MakerDAO. And then shortly after you had this epiphany about The Graph in 2020, realizing that it has tremendous utility and now you’re full-time is an Indexer at The Graph. My question is, how important do you think The Graph is for web3.0, going back all the way to that epiphany moment in 2020, that light bulb moment, how has your perspective of the importance of The Graph of web3.0 evolved to today?

Marc-Andre Dumas (01:02:51):

Well, a fundamental aspect of web3.0, which tends to be overlooked, that it’s designed to enable developers and publishers to provide services and information without running their own servers or paying third parties or cloud services for it, right? Instead, the infra must be run by the users, ideally by the users and for the users using decentralization.

(01:03:17):

In this context, The Graph fills a very important hole by making blockchain data available. And again, maybe with shell-proof and zero-knowledge proof, The Graph might be able to do a lot more in performing off-chain data processing and execution that could be useful for more advanced website or applications.

(01:03:43):

I don’t think The Graph will do everything though. There are other projects I can think of, IPFS, Filecoin that focus on storage for instance, and that probably that’s a very type of different challenge, different from The Graph, but by its position right now, The Graph today is able to solve many, many important problems in web3.0. And I think it has a vision and has the potential to solve many more problems.

Nick (01:04:09):

Well Marc-Andre, now we’ve reached a point in the podcast where I’m going to ask you the GRTiQ 10. These are 10 questions I ask each guest of the podcast every week in the hopes that listeners can learn something new, try something different, or achieve more. So are you ready for the GRTiQ 10?

Marc-Andre Dumas (01:04:23):

Yeah, let’s go.

Nick (01:04:35):

What book or articles had the most impact on your life?

Marc-Andre Dumas (01:04:37):

It’s a kid’s book, the Beginner’s Computer Handbook published in 1983.

Nick (01:04:43):

Is there a movie or a TV show that you think everybody should watch?

Marc-Andre Dumas (01:04:47):

Nobody should be forced to watch anything, but I definitely recommend Six Feet Under the series because it’s an exploration of human nature and in general, and grief and emotion and has great dark humor.

Nick (01:04:59):

If you could only listen to one music album for the rest of your life, which one would you choose?

Marc-Andre Dumas (01:05:03):

Completely unknown to your listeners, but it will be an album from Jean Leloup, a fantastic musician and songwriter from Quebec.

Nick (01:05:09):

What’s the best advice someone’s ever given to you?

Marc-Andre Dumas (01:05:12):

Don’t try to please everyone.

Nick (01:05:14):

What’s the one thing you’ve learned in your life that you don’t think other people have learned or know quite yet?

Marc-Andre Dumas (01:05:18):

Nobody knows what they’re talking about.

Nick (01:05:21):

What’s the best life hack you’ve discovered for yourself?

Marc-Andre Dumas (01:05:24):

Live in a walkable neighborhood. It improves your social life, your health, and you’ll need to buy a second car every few years so you’ll get away from that financially.

Nick (01:05:35):

Based on your own life experiences and observations, what’s the one characteristic or habit that you think best explains how people find success in life?

Marc-Andre Dumas (01:05:45):

Curiosity, that’s given to everyone.

Nick (01:05:47):

And then the final three questions are complete the sentence type questions. The first one is, the thing that most excites me about web3.0 is …

Marc-Andre Dumas (01:05:55):

Fulfilling the original vision of the internet.

Nick (01:05:58):

And how about this one? If you’re on Twitter, then you should be following …

Marc-Andre Dumas (01:06:01):

Chris Burniske, an OG crypto investor. Always great balance and print of views.

Nick (01:06:07):

And then the last one, Marc-Andre, I’m happiest when …

Marc-Andre Dumas (01:06:10):

I’m surrounded by friends and family,

Nick (01:06:21):

Marc-Andre Duma, thank you so much for your time. You’ve been very generous and provided a lot of insight for listeners and fresh perspective on what’s going on, not only with Indexers but in the MIPS program. If listeners want to learn more about you, your Indexer operation, what’s the best way to do it?

Marc-Andre Dumas (01:06:39):

Well, these days follow me on Twitter. It’s Marcandu, M-A-R-C-A-N-D-U on Twitter. Ellipfra also is present, Ellipfra is E-L-L-I-P-F-R-A. And I’m always on The Graph Discord. I’m Marc-Andre Ellipfra there. You can look me up and you can contact me. Always happy to answer questions and guide newcomers to The Graph.