Nintendo Labo developer interview – Part 1: The Concept

Mr Kawamoto, Mr Sakaguchi, Mr Ogasawara, thank you for making time for me. I’m really looking forward to hearing about Nintendo Labo.

Thank you for having us.

Let’s begin with discussing the Nintendo Labo announcement video.

Consumers had a lot of different reactions to the announcement. What was it like watching the reactions come in?

The response was really something, and I’m very grateful for it. The truth is that I didn’t know what I would do if the reaction had been something like, “What is this supposed to be? It’s just cardboard!” Luckily that didn’t happen, and the reaction was more positive than I could have hoped for.

There was a clip in the announcement video of the Toy-Con Piano lid being opened to show that it was empty inside.

We made it this way because I can’t help but feel like there’s something really interesting about all that sound coming out of an empty Toy-Con creation. We didn’t include any explanation in the video, so I was concerned that people might be confused by that shot. In the end, though, people responded very positively to it, which was a big relief.

Before the announcement went out I felt a mix of excitement and anxiety. Of course, we all felt that we’d made something interesting, but we also knew that it was a very unique, peculiar product. I was surprised – in the very best way – with how positive the reception was.

Excellent. Moving on then, I’ll jump right to it: where did you get the idea for Nintendo Labo? I suppose it’s a basic question, but I’ve been wondering for a while, so I thought I’d just ask you directly.

Not wasting any time, are you?

I have always enjoyed thinking about interfaces, and I’ve always wanted to make things that you can interact with.

I worked on the Home Menu UI for Nintendo 3DS, as well as the terrain system and player control scheme, both are connected indirectly, for the Splatoon game on the Wii U console. All of these projects involved control-and-response relationships – it’s an area I enjoy working in, and I think I might even be good at it.

Anyway, getting back to what gave us the idea for Nintendo Labo… I knew I wanted to make something new, and I knew that there are already a lot of videogames out in the world. Honestly, I just tried not to overlap with things that had already done.

We’d been thinking a lot about how to make something easy to grasp, something that could only be done with the Nintendo Switch console and its Joy-Con controllers. I just followed the logic of that question as far as I could, and it turned into Nintendo Labo.

So you started by wanting to make something different from other games, something you could only do with Nintendo Switch?

I started thinking about what makes Joy-Con controllers different from other game controllers. At first I thought it was the way the separated controllers could work together as one unit, but of course we’d already done that with the Wii Remote and Nunchuk, so that couldn’t be the standout feature. Then I thought about how unique it was that the Joy-Con controllers could be split, so two players could use one set – but then I remembered that the original Famicom had a similar setup, so even though I think it’s a wonderful feature, it’s more of a throwback than an innovation.

So I kept thinking about the controllers and what made them special, and eventually I realised it was the sensors – both the left and right controllers have sensors in them.

If we made something with both Joy-Con attached to it, we could use the differential between their sensors to do all kinds of new things…that was really the start of it.

When you say, “use the differential”, what do you mean?

Well if you had one controller be a “sword” and the other controller a “shield,” that’s an additive relationship, by which I mean each controller can perform its action on its own. If that’s the case, you could theoretically just use one controller and switch the control schemes with a button press. But using both controllers and controlling something with “the relationship between the controllers” is a control scheme that absolutely requires the use of both.

For example, the fishing game we’ve made here uses this “controller differential control scheme”.

By attaching the right Joy-Con to the handle of the fishing rod and then attaching the left Joy-Con to the reel, it allows us to use the position of the handle to measure the rotation of the reel.

Or to phrase it differently, if you were to only use the left Joy-Con, you would not be able to differentiate the motion of the fishing rod from the movement of the reel. However, if you then added the right Joy-Con and compare the differences seen from its sensors to those of the left Joy-Con, you can make it work. As we got deeper into all our experiments we started to think of the Joy-Con as bundles of sensors – that really freed us to consider all kinds of possibilities.

That’s right. All throughout development, you were always saying “bundles of sensors”.

Exactly. At the beginning of the process I didn’t realise it, but the Joy-Con are sensors before they’re controllers.

So we had these two small bundles of sensors and the hardware they attached to – a very unique design, and certainly something no one else could imitate, even if they liked the idea. That was our advantage going into the project, but not our only one. We also have the IR Motion Camera, don’t we Ogasawara-san?

Ogasawara-san, coming from the hardware side of things, how were you involved in this project?

Before we continue talking about the Nintendo Labo project planning, it might help to quickly go over the design process for the Nintendo Switch itself.

Nintendo Labo utilises the IR Motion Camera in a variety of ways. I oversaw the development of the IR Motion Camera when we were originally deciding the specifications of the Joy-Con hardware.

I see. So you were involved with the implementation of a key feature used by Nintendo Labo.

That’s right. When we were designing the Joy-Con controllers we didn’t actually have any concrete plans for the use of the IR Motion Camera at the time. So I had to think of all kinds of possibilities for it, concepts for ways it could be used, which I then presented to Kawamoto-san, the Nintendo Switch director.

It’s true. At the time I never could have imagined it would end up being used like this! When we were designing the Nintendo Switch we had all sorts of brainstorming sessions with both the hardware and software developers. It was through this exchange of ideas that we settled on the final design for the system. Nintendo Labo uses the Nintendo Switch system’s more unique design elements in a deeply satisfying way.

The three technologies that Nintendo Labo uses most are the dual gyro sensors, the IR Motion Camera, and the HD rumble function.

Nintendo Labo is immediately different from other games because it starts with the customer building something, and it’s also clear that the product is only possible with Nintendo Switch, but how did you go about turning this idea into a product? What sort of difficulties did you face along the way?

Honestly, at first, I thought it was going to be really easy.

(Laughs.)

I figured that building and repairing these things would be flexible and easy because we were dealing with engineered cardboard sheets. But once we decided that we wanted to design the Toy-Con creations so that you wouldn’t need any scissors or glue to assemble them, the project suddenly became very difficult.

You mean that you don’t need anything aside from what’s in the kits to build them?

That’s right. We thought it would be disappointing if you bought a product and then realised you didn’t have everything you needed to complete it. I mean, of course that would be disappointing. We decided we couldn’t let that happen.

We decided on this very early in the process, because if tools were necessary we’d have to include them in the kits – but the more we thought about it, the more we realised that we wanted to design the product so that you wouldn’t need any tools at all.

The idea that tools shouldn’t be necessary is what caused the design to take shape. Once we had preliminary designs to work with we did some consumer testing in the U.S. and in Tokyo. The tests didn’t go over very well, though. It was…it was rough.

I was so upset I went back to the hotel room and cried a little.

I’m serious! I was so sad! (Laughs.)

(Laughs.) You thought you’d made something anyone could build, but it turned out they couldn’t?

That’s pretty much what happened. Going into it, I thought that people might have trouble getting it exactly right, but as it turned out people had a lot of trouble with the making part.

I was mostly worried that people might think they’d built it correctly, even though it was all wrong.

They would think it was correct because they put it together themselves?

I have a distinct memory from this test session, where a child was saying, “I had no trouble making it, but younger kids might.” That’s a nice sentiment, but truthfully the Toy-Con House he made was a mess.

I knew then that this could be a problem. If people were messing up but not realising it, then they’d misunderstand the product, which is definitely a situation I wanted to avoid.

I assume you ran these prototype tests quite a few times, but what state of development was the product in at this point?

At the time, we thought it was essentially a final version.

Is that so? Ogasawara-san, you were tasked with the design of the cardboard sheets at the time, weren’t you? What did you make of this feedback? I’m assuming you had never worked on cardboard design projects like this before.

Well, we had experience designing product packaging using cardboard, but this was the first time I’d ever tried to make a cardboard design that was easy to assemble for the consumer. Despite this, I continued the working on the designs and – well, the consumer test was a real shock! (laughs)

Sakaguchi-san said it made him cry, but to tell you the truth things were rough over in hardware development too.

It was surprising because it wasn’t like we hadn’t put a lot of thought into the design at that point. We were always calling over co-workers unfamiliar with the project and seeing how well they could make the Toy-Con projects we designed, and those experiments had always gone well. So basically, consumer tests on adults had all been good up until that point. Going into the test I’d been thinking that 70% or 80% of the kids would do well. Unfortunately, that wasn’t the case. It was a real disaster.

At first the concept was built completely around the message, which was that these are toys made of cardboard. So the original designs looked less like the objects they were modelled after, and looked more like, well, cardboard. I remember talking about the Piano once, and we considered making the lid portion look more like the curves of a grand piano. At the time, we thought that customers could make the toys look like whatever they wanted, so we’d leave as many design elements up to them as possible.

For the same reason, we didn’t originally print anything on the surface of the cardboard. It soon became clear that without printing people got the front and back of the parts confused, and the build process started to fall apart. Through our consumer testing we realised that if the Toy-Con creations didn’t have a clearly-defined design, then it was hard for the customer to picture how they were supposed to look when completed.

We now have a rule that at least 50% of the surface of the specially-crafted cardboard sheets must be printed.

Did you also change the instructions or the build process?

Yes, and dramatically. We simplified the steps and reconsidered the difficult sections. We also had to find ways to increase the durability. We decided to prioritise simplicity over the cool-factor of the designs.

We did things like taking octagonal parts and changing them to hexagons. The insertion tabs went through more than ten redesigns, focusing on making them easy to insert.

So we worked on the tab designs, on making the printing clear, and on simplification. The most basic aspects of the process were making the front/back clear and the tabs. Once we had those designs worked out, we tried to simplify everything by reducing parts and so on.

There were also places that almost required three hands to pull off, so we simplified these steps. There were times where it was difficult to proceed without someone stepping in and holding a part in place for you, and those parts would be difficult for small children.

So you reworked the designs to make them simpler and easier to understand?

We decided to bring in children from primary schools and run these child-focused consumer tests to refocus the design process. We had more than 100 children participating in these.

The development team performed these tests?

Right, we figured we’d get ourselves in real trouble if we didn’t!

Plus, we could revise our designs after one test and then see the reaction at the following test to judge how effective the redesign had been. At the time we knew we wanted to make an interactive instruction manual that incorporated video content, but we didn’t have final designs of the cardboard sheets to work from. So we took photographs and slides and made little flipbooks that showed the instructions, then used these during the tests to see what parts needed improvement.

Simpler flipbooks were around 1,000 pages, while complex ones were 3,000. Whenever we redesigned it we had to take all-new photographs, too.

That sounds really daunting!

None of us had ever tried to make cardboard designs before, let alone projects that were easy to assemble, so it was an iterative process of figuring things out as we went. Still, the ease of assembly was absolutely essential to get right here, so we wanted to make all the improvements we possibly could.

People from both hardware and software development participated in these tests. The feedback we received from these tests directly led to the creation of our interactive instruction manual.

We put a lot of effort into getting it right. Many instructions can be hard to follow. But the Nintendo Labo “Make” instructions are done in an interactive video format, so the user can zoom in and rotate objects to get a clear idea of what needs to be done. Plus, the instructions themselves were written by the same writers that work on our games, so they are a lot of fun to read.

We think it’s the best instruction manual in the world!

When you hold a Toy-Con creation, it clearly feels different than other game controllers, and there’s a feeling of feedback from it. Was this something you intentionally sought to create?

We call it “physical feedback”. The Toy-Con Motorbike accelerator really shows it off.

It’s made with rubber bands so that more force is required the further you try to twist it.

Had we tried this with controllers we’ve made before, like the Wii Remote, you’d twist it through the empty air and not feel anything. But if this is supposed to be the handle, then you want the shift lever to be here too, right? And if this is supposed to be a motorbike, you want to feel the acceleration in your hands.

It’s great how the handlebars and brakes return to their starting point after the player moves them.

I don’t think I’m the only one that wants physical feedback in their hands. It’s much more immersive than waving something in the air. With Nintendo Labo, we were finally able to build that physical feedback into the system.

By adding constraints to the two Joy-Con, by making it so that they can only move in certain directions, it’s very easy to get that value from them. The restriction of movement makes it so the input values are limited – in a good way. If you make the controller shaped like a fishing rod then it can only be held like a fishing rod, so consumers will intuitively understand the correct way to hold it. I’d run into this kind of problem before.

I remember when we were developing Splatoon. We had the opportunity to show what we’d been working on to people outside of the development team, and we instructed players holding the Wii U GamePad to “tilt to the right”. As it turns out, everyone had their own interpretation of what we meant. I particularly remember one person that just slowly craned their neck to the right…

That caught me off guard, but it drove the point home that once these products are out in the world, there might be a lot of people with differing ideas of how to use them, of how they were intended to be used. The possibility for variance there is a scary thing. Once it’s out of our hands we can’t see how customers are playing with it, you know?

That must be intimidating for a developer.

Oh, it’s terrifying.

First, we have the cost of explaining how it works to the user, the “study cost”, if you will. If we can’t get the user past that, they can’t play any games at all. So we want the explanatory matter to be as short and simple as possible. We focused on this a lot while developing Nintendo Labo, and we realised that to make something easy to understand it had to be immediately clear how it was held, and how it was supposed to move.

I see. That makes sense.

And yet… You know the antenna we attach to the Nintendo Switch as part of the Toy-Con RC Car?

Yes, it’s there on the top.

It actually doesn’t serve any function at all.

We tried to get rid of everything that doesn’t serve a function, and spent a long time removing unnecessary parts. But this antenna, it doesn’t do anything. We just like it. (Laughs)

But it feels so different with the antenna on there!

I think so too, and we realised that we had to rethink what we meant by serving a function.

You don’t have to explain what it is. You know it’s an RC Car just by looking at it.

I’ve been wondering about the Toy-Con Fishing Rod. Why did you make it so the reel clicks when you wind it?

Don’t you think it’s better with the clicking? (Laughs.)

But seriously, we did it because the sound is different depending on the direction you wind it. It just kind of worked out that way. (Laughs.)

And this part here (points to top of the Fishing Rod) doesn’t really need to extend.

It feels very different if the Toy-Con Fishing Rod makes a sound, and fishing poles feel like they should be extendable. (Laughs.) And the Piano keys bounce back to the starting position after you hit them, don’t they?

That’s right. Right from the start of development we were really concerned with the weight of the keystroke. It had to feel right. It’s really at the core of what Nintendo Labo is all about. We intentionally set out to make experiences that weren’t possible with a standard game controller – I think that was what really guided the development process.

We feel the same way.

Many parents have expressed interest in Nintendo Labo because they see educational value in it, particularly in the Toy-Con Garage mode. From what I’ve been hearing, it doesn’t seem like specialists from the academic sector were brought in to consult on development. Can you talk a little about the educational aspects of the product?

You’re correct that the development of Nintendo Labo was handled internally, and we didn’t intentionally aim to make an educational product. However, we did set out to make Toy-Con Garage as easy to use as possible, so that even young children can set up a project and experience the joy of seeing it work. The Toy-Con assembly process and the software were both created with this goal in mind.

We designed the games included with each Toy-Con to show off what a professional can do with the system. You don’t need to jump into Toy-Con Garage to have a deep experience with Nintendo Labo. Simply building the Toy-Con projects in the kits and following along with the instructions is a fascinating experience. Building and discovering how the Piano works, for example, is really intriguing.

So the “Discover” portion of Nintendo Labo is fully explained in the software?

Right. I also want to emphasise that one of our goals was to explain how the technology behind the Toy-Con creations works, so we've designed the discover portion of the software to be as user friendly as possible. An interesting etymology tidbit: I believe the word “discover” originates from the idea of removing the cover from an object.

For example, you can use the Nintendo Switch console’s touch screen to view transparent versions of the Toy-Con creations and Joy-Con, which are a lot of fun to play with. We designed the Discover portion of the software so that you can enjoy the character interactions even if you don’t understand what’s being explained.

Truthfully, our highest priority was to make something interesting and fun. We wanted to make an experience that helped people see that discovering how things work is fun in-and-of itself, and that making things is rewarding.

And it’s not just discovering and making, the playing is fun too!

Yeah. (Laughs.)

I think that if we want to get people to enjoy the making and discovery portions, the play aspect of the product needs to be a lot of fun. It’s a very important part.

We can’t speak with authority on what makes something educational or not.

We aren’t education specialists, after all.

So rather than calling it an educational product…

We really just want our customers to experience the joy and fun of making things.

I have personally learned so much from games over the years, so I hope that everyone that gets the chance to experience Nintendo Labo finds something valuable they can take away from it.