Scaling Innovation

It’s been a long time since I sat down to write out my thoughts. Over the past couple of years, I’ve been working on my MBA. I successfully graduated and learned a lot about business in the process. Today, I want to write about something I’ve been working on over the past few years. I want to talk about how we’ve scaled innovation – from a single product to now looking at nearly a dozen. These are not features built on top of an existing product but multiple products with wildly different but similar requirements.

What does innovation mean?

When discussing innovation within the product engineering space, it may not necessarily mean that we’ve created a novel technique to solve the (technical) problem or invent new languages or frameworks. Innovation within the product engineering space combines product, business, design, and engineering to co-create new products and businesses or iterate (slowly innovate) on a fundamental idea or concept.

I’m not excluding building new languages or frameworks if they enable innovation. For example, when Facebook built React, it enabled them to build features across multiple teams quickly. Or when Google created Golang to scale and build robust concurrency across multicore machines. These innovations have helped businesses move fast and create profit.

But you don’t need to be Google or Meta (Facebook). You can innovate with your business and product to solve other people’s problems. I don’t need to look far to see innovation within Indonesia. Take Gojek as an example. It started as a ride-hailing company and has moved into food delivery, last-mile delivery, payments, and streaming. Not all its product innovations have succeeded. A company needs to know when to drop a product. An example of a dropped product/venture is GoToko. It’s hard to compete against Mitra Bukalapak (and the non-tech supply chains that already exist).

Who are the Intrapreneurs?

An intrapreneur is a new-ish concept that I’ve come into. On the surface, you can consider this person to be an entrepreneur-in-residence. However, there’s some nuance to this. As an intrapreneur, you have access to funding and resources straight away that you would not have elsewhere. However, you probably won’t get equity in your business venture (or very little).

When I think about an intrapreneur, these people work on new business ideas that may compete with, complement, or extend the original business. For example, look at AWS as one of the biggest incubators there is. We see them constantly release new products that compete with or complement their existing set of features. I’d imagine each of these products has its business metrics. But that’s not the only intrapreneurial-spirited company I’ve seen. Look at ByteDance (owner of TikTok). They’ve released business-to-business platforms and looking at publishing books. The breadth and speed show how they use their data and platform to innovate.

As an engineer, you can be an intrapreneur. If you’re one of the early founders of these new products or businesses, you’ll be involved in the planning and strategic direction – as any other technical co-founder would. You’ll make tradeoffs with your technology choices as you choose speed, quality, or scope. It’s not just the business leads or product managers who can be the intrapreneurs.

Where do you start?

If you’re a product company offering (physical products) toys – you wouldn’t change your business model to now offer a SaaS (Software as a Service) for finding cleaners in New York. It’s the same for Bukalapak; as an e-commerce company, we wouldn’t start a SaaS platform for cybersecurity. We don’t necessarily have the expertise, and even if we did – how could we continue to compete in both e-commerce and cybersecurity SaaS? The answer is – you probably can’t. The two companies and products are too different to allow for intrapreneurship.

So… then, what? You need to be an intrapreneur within the same space as the company operates. For example – going back to Gojek – we can see ride-hailing is about the last-mile delivery of people. What else can you deliver? Food? Groceries? Packages? Online e-commerce (Tokopedia)? How can you use this fleet of people which is your moat? Can you build a supply chain using idle riders? Do you have idle riders? How much can a rider carry? Can our riders offer services? Is there something else besides ride-hailing adjacent to what they’re doing? Payments? Moka POS? Midtrans? What could offer more margin? Cloud kitchens?

At Bukalapak, we’ve decided to innovate using specialized marketplaces. You can find this as part of our annual statements. What does that mean for innovation and intrapreneurs? It means that we are looking for underserved niche markets. Gaming is a big one for us. What other markets can we experiment in? What’s working overseas that we don’t see in Indonesia?

All it takes is an idea. You pitch it to whoever may be the relevant decision maker. Ensure you have data and reasoning behind it. State how you want to be involved in the product.

For Bukalapak, our Incubation team started with just one product manager and one engineer. Eventually, we’ve grown it into two tribes within Bukalapak, with over fifty on the engineering team (hired internally and externally). Our composition changes quarterly (sometimes more frequently) depending on each business/product’s lifecycle and needs.

Getting buy-in

How do you get buy-in? How do you convince your boss (or boss’ boss) to invest in your idea? The TLDR (too long, didn’t read) answer is data. You’ve researched the market size. You’ve done the competitive analysis. You’ve got a budget and have the following year’s forecast. You know how much you need to spend on marketing to get traction.

But do you need everything up front? Can you just run an idea up the hierarchy on a hunch? Ideally, you won’t just suggest an idea. It’ll be forgotten or pushed aside. If you think about it, why would a company invest in your idea if you don’t know how big it’ll be or how long it would take to develop? Especially public companies will be cautious about risk – how much do they risk by investing in your idea?

Incubation lifecycle

During our many discussions over how best to support new businesses, we started to define our businesses’ stages and how we support them. Unfortunately, this is an ever-changing problem when you have limited resources and prioritization from the group.

Research

In the research stage, the business team mainly researches the market, margins, and potential revenue and growth. Product may be involved here to help shape the direction and product, but mostly this comes from strategy. We want to be reasonably sure before investing our time (and money) into a problem.

Where do ideas or problems come from? This is a tricky part to answer as there is no one place where we’ve adapted ideas and run with them. We’ve had ideas crowdsourced from the entire company. We’ve had partners of ours suggest ideas that we’ve run with. All it takes is one passionate person to do some initial research and ideation before we can see the potential. These are the intrapreneurs I mentioned earlier—the ones who will see the idea through.

In the research phase, there’s not a lot of engineering. We do have some exploratory engineering that we have done to size costs or to give a rough picture of how long it will take (investment for the company). Generally, we will not build anything until we are sure we want to move to the proof of concept.

Proof of Concept

We explore the idea in the proof of concept (POC) stage as quickly as possible. Since we’re an e-commerce company, that usually means using existing e-commerce platforms and doing as little engineering effort as possible (because we’re expensive). You may use platforms such as Shopify or Order Management Systems like SmartSeller. We aim for the fast delivery of the POC to validate the idea. We’re not here to validate any new technology or anything scalable. The more manual work, the better, as we know the pain points.

Sometimes we cannot get away as quickly as we like with the POC. Sometimes we have to build something. This can be especially true if we know for sure we are going to invest in this direction. However, I condition the team to make as many tradeoffs as possible. We want to be scrappy. We don’t need any fancy frameworks if we don’t need them. Just build. Testing? Let’s do lightweight testing to ensure we don’t deliver a pile of poop. Tracking? Data? Metrics? Business metrics are the most important here. We want to know how the product is performing. Engineering metrics can come later when we start to scale. But what about reliability? What about scalability? Does a product with ten users need to have the scale of a product with one million users? Will our customers notice if we’re down for an hour? Are they more forgiving of new products?

Ideally, we want to get the POC out within one month of the word go. We want to iterate on customer feedback. It’s not always possible to get something out that fast – but as a business, we try – and we’re still sharpening our knives as we learn.

Minimum Viable Product

At this stage, we’ve already established that people will use our product. We’re committed to investing in it further. Here is where engineering usually starts with customization and solving the problems with the current POC. We’ll likely spend a couple of quarters building and enhancing the MVP to ensure a solid product market fit.

Once we hit MVP, we need to figure out how to reach the next growth stage, the scaling stage. Do we need to add more features to ensure we know our product-market fit? Do we need to scale operations and add more locations (if physical)? There are many questions we need to ask at this stage.

It’s still a young company; we must decide on the business metrics. What will we look to for scaling a business? Is it growth at all costs and figuring out profitability later? Should we figure out how we will get profitable now and create a way to sustainable growth? What metrics will we track, and what will be important as our objectives going forward?

We’ll also start to look into the organizational structure of the business. How does the product team work with business/operations/marketing? Where do we need dedicated people, and where can we use centralized teams?

Scaling

Now that we have the product in place – we need a dedicated team to help scale it. We’ll likely add features we know will hack the supply or the demand. Operations will likely be heavier than engineering as they will scale out faster (unless you’re building a SaaS).

From the engineering perspective, we’ll be upping our game. We’ll look into DORA (DevOps Research and Assessment) metrics, ensuring sustainable and productive engineering practices. We’ll be working on our test coverage, as the more customers and revenue we have, the more an outage affects the bottom line. We’ll have proper documentation coming through to help future engineers understand our systems and the decisions we’ve made.

Established

If you were to look into the company’s annual statements, this product would be big enough to warrant a separate financial statement breakdown. We’ll likely have an entire tribe dedicated to this category (or more), and we’ll be focusing on optimizing and growing the business sustainably.

Our ways of working will follow established practices, and objectives will be clear and concise. We know what we want to achieve and when and results are largely predictable. Multiple squads of engineers work on different aspects and problems within the product’s ecosystem, and there is enough revenue to support the research and development efforts.

We’ll be looking to the market to see where we can better improve our competitive advantage (which we likely have) and where we can grow further. We’ll have established our unique value proposition (although that should’ve been done during the research phase).

This state is our end goal for each product. We want them to be big enough to support themselves and other parts of the group. But we won’t stop here. We’re always hungry and looking for the best opportunities.

Engineering for (all) scale

So – how do you build something to enable everything from a single user/customer to millions of customers? Is that possible? At Bukalapak, we’ve done just that. But it hasn’t been easy, and it’s not perfect, but we’re working on it. How does it look?

Lego blocks

One of the best analogies I’ll steal from our SVP of Engineering is imagining you have some Lego blocks. You can build different things with these blocks, but you’re still using the same blocks as everyone else.

Before we get into the nitty-gritty of the architecture, I want to point out that we didn’t invest in the platform for the sake of a platform. We invested in the platform as part of building one of our more significant incubator investments. This product got a lot of attention while product engineering set to work across the company building out the Lego blocks to make it happen.

When considering an e-commerce model, we know what needs to be there, right? We need to have products, a list of SKUs, and categories. Pretty much every e-commerce (or ORM) platform will have the basics of this. We’ll also need some way to make an order – i.e., to purchase the product on display. Sometimes (most of the time), we’ll need to know the customer a bit more and have a secure way for them to check their orders. Lastly, we’ll need a way to update the customers on the status of their orders.

I hope you can see where I’m going with this – especially when you have multiple squads (teams) and tribes wanting the same thing. Yes, we used a microservice architecture to create the domain layer blocks. Below the domain layer, we have the infra layer, a set of reproducible (cloud) infra, and security boxes ( to also allow for multi-cloud). Below that, we have the support teams who help hold up and support the product engineering teams.

Above the domain layer, we have our business layer. In a classical monolithic approach, this would be where all the business knowledge is held. We use the same approach in our coordinators – which takes the building blocks from the domain layers and adds the specific business processes and knowledge. When we use our domain layers, we have two concepts product engineering teams can employ. Reusable or shared.

A reusable domain is one where the product engineering team deploys it, and there might be business-domain-specific additions. For example, if we want SKUs for used electronic goods, we will want our suppliers to state what the state of the product is and have that as a child SKU of the parent (as you wouldn’t want to ship the wrong electronics good – hence each is a unique SKU).

Shared domains, on the other hand, are domains that can be shared and multi-tenanted. An example of this would be identity. We can create tenants within the identity service, share the capacity (to reduce costs and support), and ultimately reduce development time and effort (there’s a whole industry dedicated to this specific example).

All these Lego blocks aim to reduce the effort needed to create a new e-commerce business. By standardizing and making things boring, we hope to speed up our iterations and experimentation with businesses (and reduce investment costs). Imagine if we told you we would launch a complete MVP of a business next month (with most of the bells and whistles you require).

Initial support

Let’s say we’ve created all the initial domains and launched the first product. The business is asking you to replicate this across several other products. What happens to the domains? Who supports them? What happens to bugs? Let’s answer these questions one by one.

What happens?

Originally, when we first launched the domains under one of our businesses, we ensured that the domains had support, i.e., a dedicated squad. For efficiency reasons (as well as practical reasons), at Bukalapak, this didn’t last long. We’re a product company and not a platform company like AWS. The domains were never going to be a priority. Their job is to make the development of e-commerce products faster and not to be the main objective.

So what happens to the domains? I think of our technical, organizational structure like Saturn (yes, the planet). In the center, we have the planet: all our shared resources like core squads, security, infrastructure, IT support, technical cost operations, architecture, etc. In the rings, our product organizations are kept there by the gravity of the central teams. Our product engineers rely on central functions to ensure consistency across the organization.

The domains moved to our planet, our core teams for maintenance and minor bug fixes and security patches. However, if you need (as a product engineering organization) logic changes on one of the platform domains, you would use a model called inner sourcing. It’s like open source (whereby anyone can contribute) but internal to an organization. This allows our core teams to be lean and focus on the domains’ reliability, security, and maintainability while the product engineering teams can focus on just using them.

Our current support model is not without limitations. For example, how do the core teams prioritize domains or services? How do they know which business to support? Which product is most important to the company’s goals? What’s the SLA for pull request reviews by the core teams? A lot of negotiation is involved, and we’re still working out the details.

Scaling Innovation

Narrowing this down to a few words is easy – build a framework and platform supporting business needs and growth. However, this won’t give you the silver bullet, as every business differs. What it takes is individuals who are intrapreneurs and who are willing to try (and fail). There needs to be that culture in place for innovation to succeed.

For engineering, it can be hard to break the chains that tie you down when you’re in a large company. You need permission to try something different and break the processes slowing down you (and your team). Be agile (not with a capital A), and don’t fear the unknown.

I’ll leave you with the quote below. Failure is the key to successful innovation. You’ll have products that succeed and products that fail. People should not be punished for their failures, but we should treat them as learning opportunities.

One who fears failure limits his activities.

Henry Ford