Chris Lattner on the origins of Swift

Updates:

  1. Feb 20, 2019
    Fixed Brian Lanier's name, which I had misheard.
  2. Feb 23, 2019
    Added two footnotes with clarifications from people who were there.

In episode 1 of the new Swift Community Podcast, Chris Lattner, Garric Nahapetian, and John Sundell spoke about the origins of Swift and the current state of the Swift community, among other things.

This is a transcript (edited for readability) of the parts I found most interesting. You’ll see I mainly quoted Chris Lattner because I think his account of how Swift was created is the most relevant to preserve for posterity. That’s not to say that what John and Garric said was any less interesting. You should really listen to the entire episode — it won’t take much longer than reading this post, anyway.

The Swift Community Podcast is also worth checking out on its own. It’s very much intended as a project where anybody can contribute in various forms (the three go into more detail in episode 1). In this spirit, I worked on creating the transcript in the open, and even received help from the community in the form of code and in editing the machine-generated transcript. Thank you to everyone who helped!

You can find the complete transcript on GitHub (in WebVTT format). The entire podcast is licensed under CC-BY 4.0.


On the origins of Swift

(Starting at 16:59)

Chris Lattner: I have to go back in time to WWDC 2010. We just launched C++ support in Clang and that was a huge, huge, huge amount of effort for tons of people. And I was feeling both very happy about that, but also a little bit burnt out because it was a lot of really nitty-gritty work. And you can’t implement C++ without thinking, “gosh, there should be a better thing out there!”

And so I got into a number of conversations with a guy named Bertrand Serlet. Bertrand was the head of the software team at the time, and Bertrand is an exceptional engineer. He’s an amazing human. He’s also a bit of a language geek. And he had been pushing to make Objective-C better. He always wanted more out of Objective-C. And he and I got into a number of one-on-one whiteboard sessions.

At the time, Swift was called ‘Shiny’.

Bertrand ran all of software at Apple, so he had no time. But he always encouraged me to drop by at the end of the day and try to see if he’s available. And he’d stay late, and we would geek out on the whiteboard. We talked about a ton of different things: the goals [of a new language], weird details like the type system, and we eventually turned it into a pitch deck. So I built the pitch deck for him and came up with this idea of building a new language. At the time it was codenamed “Shiny”, as though “you’re building a shiny new thing”.1 I was also a fan of the Firefly TV show.

John Sundell: Was the file extension .shiny?

Chris Lattner: Yes, in fact, it was. You know, at the time, it was super tiny. It was really just Bertrand and I talking about it. Another really awesome engineer named Dave Zarzycki got involved in some of that early conceptual discussion.

One of the things that came out of it really early on was that we started talking about memory management. At the time, we were both convinced that there had to be a good way to solve or improve memory management and we needed to get to memory safety. So you have to have automatic memory management.

The goal of having automatic memory management was the genesis of Swift internal design discussions turning into Objective-C features.

One of the key things that came out of that was ARC and the idea that we could have the compiler own and drive this instead of having a runtime. Objective-C had this libauto garbage collector at the time, but it had a number of problems. That was kind of the genesis of Swift internal design discussions turning into Objective-C features. And many of the things that came out, including ARC and modules and even literals and things like that, really came from the behind the scenes development of Swift.2

John Sundell: So there was a bunch of features that you had in mind for Shiny and what would later become Swift. But you said, “we don’t want to wait until we have a brand-new language completely implemented. Let’s go ahead and take those features that we really want and just add those to Objective-C.”

As part of the ideation process behind building a new language, you always have to ask the question, ‘why not make the existing thing better?’

Chris Lattner: If you think about it from a different perspective, Swift may seem obvious now, but at the time it was not obvious at all to anybody, and even me. Bertrand was, and is, really great because he’s always super encouraging. And he always pushed through the doubts. Bertrand’s a bit of a scientist. He just wants to find truth in many ways. So yeah, there’s tons of uncertainty, but at the same time, there’s lots of good ideas. And so he and others were pushing, as part of the ideation behind building a new language, you always have to ask the question, “why not make the existing thing better?” And the answer was, “clearly, we should make the existing thing better”. And so ARC and all the other things that came out of that happened.

But in the case of Swift [the tentpole feature] became memory safety. And you can’t retrofit memory safety into Objective-C without removing the C. And if you remove the C you just lose too much and it becomes not Objective-C anymore.

Garric Nahapetian: Right. So was it somewhat like a Trojan horse to add those Swift features to Objective-C, which then made it easier to convince people about Swift later, as you’d already done the Objective-C work?

Chris Lattner: [There were] interesting internal dynamics. I guess we were very focused on making Objective-C and the platform better. In terms of the Swift development, it was a way of de-risking in a sense, because if you say, “we’re gonna roll out everything all at once”, and none of it is tested, then it’s hugely risky. But if you roll out “minor” things, like a whole new memory management system, separately and then you iterate, debug, and develop that with the community, then it takes a certain amount of risk away. But I would say that both the external and internal community at Apple was kind of saying, “why are you prioritizing this? It feels like we’re kind of on a random walk. Why are you doing this and not this other thing?” And so there was an interesting dynamic with that.


On growing the initial team

(Starting at 22:49)

Chris Lattner: Apple has a really strong engineering team. There was a ton of people at the time, all writing Objective-C, that are highly opinionated about things, but also have a huge depth and background in building frameworks, apps and all these kinds of things. And so there were a lot of ideas on how to make Objective-C better. There were big white papers written by luminaries that had been working since the NeXT day on this stuff. And so there was a tremendous internal community that drove that.

At the time, Bertrand and Dave and I talked about ideas, and I started coding a prototype of the compiler. But obviously, I could not build everything myself. And so what ended up happening is, around April 2011 we discussed Swift with management and then got agreement to pull in a few more people. This is when folks like Ted Kremenek, Doug Gregor, and John McCall, and a bunch of other exceptional engineers over time, got pulled in. And looking back on that, it was really interesting because this was the first time that other language and design experts were taking a critical look at this. And they had a lot of really harsh feedback. It wasn’t intended as harsh, but they were right. It [the language] was terrible at the time.

And [having] the ability to have one of the world’s experts on generics pulled into this, and the team that had built the Clang compiler and had been working together for years on many different, really interesting projects, being able to draw on that engineering talent was critical to making everything happen. And these are just a few of the people that helped drive and build that. But it was a big deal.

John Sundell: What was the state of the language at that point? Like, what did the syntax look like? What part of the compiler infrastructure was there? Was it all very much in the prototype phase or had you gotten a little bit further along at that point?

Chris Lattner: It was very much in the prototype phase. This is all public because the revision history is all public. There’s a changelog that goes not all the way back, but quite a ways back.

Before Doug got involved, there was no generics system. We wanted the generics system, but I don’t have the expertise to design it myself. And Doug is the guy. John, I remember very early on, took over making actually generate code instead of just being a parser. That was a major piece that he took on.

There were tons of missing pieces, but there are also pieces that go all the way back. I think var and func go all the way back to the beginning. Some of the basic syntax ideas are very, very similar [to what you see in Swift today].

When building a new thing, often the ideas are ahead of the documentation and the documentation is ahead of the code. It was very much like that. And the ideas are so way ahead of the code right now, incidentally.

So it was very much a prototype. But a lot of the ideas [were already there]. When building a new thing, often the ideas are ahead of the documentation and sometimes the documentation is ahead of the code. It was very much like that. And the ideas are so way ahead of the code right now, incidentally.


On Craig Federighi

(Starting at 26:10)

Chris Lattner: Another part of the community that was really important was a guy named Craig Federighi. Craig is well known in the Apple community. He became involved in the project some time in early 2011. That was right around when Bertrand retired from Apple and Craig took over his job.

Now, Craig is a really, really interesting person. He is super charismatic, both on stage, but also in person and in one-on-ones. But the thing that I think a lot of people don’t understand is that Craig is just ridiculously smart. And he is super deep on so many topics. And I didn’t expect this, but he knows a lot about languages. In former roles, he’s worked with Groovy and lots of other kinds of languages and things that I had not come in contact with. And he’s not just the high-level guy who cares about strategy. He also cares about a ton of things, like closure syntax, keywords, all that kind of stuff.

And Craig really was the hard driver and the advocate for making it real and making it relevant to Objective-C and caring about Objective-C development and caring about the APIs and caring about what the APIs imported into Swift look like, and all that. And Craig, in addition to giving great feedback, just kept, and keeps, an exceptionally high bar on the team and the project. He really helped shape a lot of what Swift is today.

Garric Nahapetian: It’s pretty cool because he was the one that first announced it on stage at WWDC 2014.

Chris Lattner: Yes.

John Sundell: And then he brought you out, right? That was the classic line: the “Objective-C without the C”.

I have mixed feelings about the ‘Objective-C without the C’ tagline, because that’s really not what it’s about.

Chris Lattner: Which honestly I have mixed feelings about, because that’s really not what it’s about.

John Sundell: It’s a good tagline.

Chris Lattner: It was the right thing to say to the community at the time.

[…]

From the beginning of the project, my goal was to build a full-stack system.

Chris Lattner: The reason it is conflicting to me is that from the beginning of the project, my goal was to build a full-stack system. It was to look at all the existing systems out there, see what’s good or bad about each of them, and then cherry-pick the best ideas from systems wherever they come from. And the goal was really to build something that you could write firmware in or that you could do scripting in, that you could write mobile apps or server apps or low-level systems code, and have it be great at all of those, not just some terrible compromise.

So that positioning was absolutely the right thing to do [at the time]. But hopefully, Swift will grow over time in kinds of what it is able to do.


On the importance of documentation

(Starting at 30:32)

Chris Lattner: [There is a] last group I want to talk about. Because, you look at what Swift is and sure, it’s a compiler, it’s a language spec, it’s a set of APIs, it’s an IDE. But the thing that makes it real and the thing that reached so many people is the work done by the Developer Publication Group. These are the tech writers at Apple that wrote things such as the Swift Programming Language book. And the success and the rapid adoption of Swift is in huge part directly related to such high-quality, good documentation and a book being available on day one. And it’s continued to be maintained today, and it’s amazing.

We pulled the tech writers right into the design meetings.

And part of that was that we pulled the tech writers right into the design meetings. Folks like Tim Isted, Brian Lanier, and Alex Martini spent a tremendous amount of time in weekly meetings where we were arguing about, “do we use dot versus colon?” Or, “do we use this keyword versus that keyword?” Or, “should we change func to def?” But also about the depths of the type system and how the codegen algorithms should work. And how do we get good performance? And how do strings work? And all the different pieces that go into it.

If you can include the explaining-it-to-people part into the design process, you get something that’s so much better.

I’ve seen so many systems where you build a thing and then you try to explain it afterwards. And when you go to explain it and it’s really awkward and you’re like, “wow, I have to explain the workarounds for this thing”. If you can close that feedback loop and include the documentation, include the explaining-it-to-people part into the design process, you get something that’s so much better.


Developing Swift is a team effort

(Starting at 34:58)

Chris Lattner: I mean, this is the kind of thing where a lot of people say, “Chris invented Swift”. And well, there’s a truth to that, I have pushed it over many years in many different ways and kind of shepherded the project. But it misses the fact that there’s hundreds of people involved in so many critical ways. Building the debugger, building the IDE, building Playgrounds, building educational content within it, building all this stuff, building the community aspect of it.

There’s so many different pieces of this that far transcend me. It very much is a community of people both inside Apple, but also outside Apple, that are all pushing together and building things and contributing in their own ways. And that is why Swift is where it is. And that is why I see it growing.

Garric Nahapetian: And that’s one of the reasons why we want to have this podcast, to bring some of those people to light.


On not writing Swift code as a compiler developer

(Starting at 42:15)

Chris Lattner: One of the things that’s fascinating is that the two of you have written far more Swift code than I have. So I know a lot about the insides and the how and why and how it fits together, but you guys have the actual experience of building and using it in production.

John Sundell: Yeah, that’s funny. There’s this bunch of people working on Swift, working on the compiler, and they’re all writing C++ most of the time. Just a quick aside, what’s that like? You have this cool language that you’ve designed, and everyone else gets to use it, but you still have to use C++.

Chris Lattner: [laughing] It kills me. It’s terrible. It is the universe’s trick on me, to force me to keep writing C++.


On community feedback

(Starting at 43:11)

Chris Lattner: One of the major reasons that Swift is good right now is that there’s a big community of people out there that were blogging about it, right? And that feedback really shaped Swift 1 and 2. That was the signal, all those complaints, people saying “this doesn’t make sense”, “I’m having this problem with this, that and the other thing”. That really shaped and prioritized and drove a lot of what got built.

Early community feedback really shaped Swift 1 and 2.

Swift 1 accidentally, well not accidentally, intentionally shipped without error handling. It also shipped without protocol extensions, things that we absolutely wanted and just didn’t fit with the schedule. So we knew we had to build those things, but a lot of what happened in that first year or two was directly driven by the community. And when Swift became open source, Swift Evolution is a hugely great thing. It’s maybe not ideal in terms of optimizing for human time spent, but it’s an essential part of what makes Swift really exceptional. And I think that is total credit to the community of people that spend their time there and that help shape and drive it.

John Sundell: One thing that comes to mind is how not only articles and content, but also open source, it feels like, has fed a lot into Swift itself. For example, with things like Codable, people were creating a thousand different JSON mapping libraries. I was one of them. I built Unbox because it was one of those things where in Objective-C you didn’t think a lot about it most of the time. You just said, here’s a dictionary, let me access this key and I’m just gonna assume that’s always a string. But once you sat down and wrote that same code in Swift, you realized you needed to build a big pyramid of if lets. So I can only imagine that seeing those things that people tried to solve in different ways must have also fed back into the design process.

Chris Lattner: Yeah, absolutely. And the design of Codable came from a framework team at Apple that were outside of the core Swift group. They were really passionate about this, and they came up with it and drove it and advocated for it.

It’s maybe not understood how much the community shapes Swift.

And community can mean so many different things, right? Look at Result. Why is Result now coming into Swift 5? Well, it’s because so many people built it over and over and over again. And while the core team really doesn’t want to have a Result type because it’s kind of a failing of the language, and when we get concurrency it won’t be as necessary. But the community says loud and clear, “look, we need it. It doesn’t matter if it’s not ideal in the long term, we need it.” So the community really does shape things. And it’s maybe not understood how much the community does have that effect.


On initial expectations

(Starting at 46:07)

John Sundell: Looking at the current state of Swift and what it’s become since you put it out there, has it matched your expectations, if you had any? How does it match the ideas that you had in the beginning now that it’s been out for a couple of years?

When Swift 1 launched, there was the question, can we capture the Objective-C community? Can we capture the iOS ecosystem, or are we going to fragment it?

Chris Lattner: Well, I guess expectations change over time. Back in 2010–2011, I had no expectation that it would ever turn into anything. And I admit that it was very much a fun side project. It was a nights and weekends thing originally. Have a day job, have a side project that’s intellectually interesting and challenging. As it became more real and it launched into Swift 1, there was a question of, can we capture the Objective-C community? Can we really capture the iOS ecosystem, or are we going to fragment it? That was a real concern. And so I’m really happy that today, I think, the vast majority of the community is happy with Swift. And while there’s still tons of work to be done, it seems like it’s very successful within that community.

Chris Lattner: But there’s a huge open frontier. Swift on the server is making great progress, but there’s still a ton of work to be done. And there’s other communities out there. I’m particularly fond of the numerics and machine learning communities, and they’re super important to the world. And there’s a ton of really interesting people in these communities, and I think Swift could be really great there.

Swift world domination is a joking goal, but it’s based on the belief that people who use Swift love it.

And I jokingly say, Swift world domination is a goal. It’s a joking goal, but it’s based on the belief that people who use Swift love it. And if that’s the case, I’d love to bring that joy to more people and help improve the world. So many of the systems out there are really unfortunate in various ways, right? People are still writing C. And just in terms of the bugs and security vulnerabilities and things like that, it’s really unfortunate. And there’s ecosystem problems, there’s other challenges that we have to overcome, and that’s something that we as a community can do. I don’t see any limits on that.


On popularizing Swift outside the Apple community

(Starting at 50:18)

Chris Lattner: While this has been a very upbeat, positive discussion about Swift, Swift also has lots of problems. And I think we need to be open to talking about that and treat that as a problem solving exercise. There are major issues with the Linux ecosystem, for example. The Windows ecosystem is barely getting off the ground at this point. There’s so many different things that really need to happen to get it to be relevant to the broader audience.

Swift also has lots of problems. There are major issues with the Swift on Linux ecosystem. Swift on Windows is barely getting off the ground at this point.

[…]

Chris Lattner: Our goal is to build an inclusive community. But if you’re not an Apple developer, it can feel alienating that every time you search for some Swift thing you end up in an iOS discussion.

If you’re not an Apple developer, it can feel alienating that every time you search for some Swift thing you end up in an iOS discussion.

John Sundell: Yeah, exactly. “Here’s how to do that in a UIViewController.”

Chris Lattner: Exactly. And that makes you feel like an outsider and it can send the wrong signal. Nobody means for that to happen, I assume, but the effect is real. And that’s a challenge that we as a community face. And I don’t know that there are good, easy answers for any of these problems. But I think we can get there.


On Swift Evolution

(Starting at 55:12)

Chris Lattner I could probably talk for five hours about this because it’s very complicated. The short answer is that one of the default views that I hold is that open is better than closed. If you can get more people involved, you will get a better thing. I think that the Swift Evolution process has lots of problems, Garric identified several of them. But I think it’s generally a really good thing. It’s also a great forcing function that slows down the evolution of the language, which I think is good. It’s better for evolution to be deliberate and carefully considered than it is to be fast.

I also think it’s good to force a certain amount of documentation and process on it, which is important. It’s also an interesting way that leads to Apple collaborating with the community in a different way by providing a form for doing that, which is really great.

I don’t see Swift Evolution as a fixed thing. It’s changed in many ways over the years and it’s always a hard tradeoff between, are you allocating design authority to the community or prioritization ability to the community? This is challenging because if you leave it up to the community completely, then you get lots of small things. But there are major things that need to happen in the Swift world, including concurrency, for example.

That’s a bigger effort, and that’s harder to do through a completely bottom-up community process. So I don’t know. I think that Swift Evolution is a really good thing. I’m really glad we have it. I agree it’s not the only thing, and it should not be the only thing, but I think it’s overall good.

[…]

I’m always on the lookout for what is the catalyst that allows the Swift package ecosystem, the Swift-on-the-server community, or the machine learning community to take off.

And each change takes time to digest in a certain sense. When a major new capability comes in, it takes time for the community to figure it out and apply it and figure out how it fits with everything else. So taking time to do that is good. The number one thing I learned from Swift Evolution is the power of having the right catalysts in the community. Swift Evolution really pulled together a community of language geeks to care about one specific aspect of the Swift project. I’m always on the lookout for what is the catalyst that allows the Swift package ecosystem to take off, or that allows the Swift-on-the-server community to take off or to get the machine learning community together and do cool things.

How do we find those catalysts that allow the people to come together in the right forum and really build on their strength and get the energy but also get the talent, get the people working together in a collaborative way?

  1. Jordan Rose shared an anecdote (thread) that’s related to the name Shiny:

    the “magic number” for swiftmodule files is E2 9C A8 0E. Those first three bytes are UTF-8 for ✨ (U+2728 SPARKLES).

    ↩︎

  2. Greg Parker clarified that ARC came only indirectly out of the development of Swift:

    The implementation pre-dates Swift. But behind the scenes the idea that later became Swift was an important driver to get management to commit the resources necessary to build and deploy ObjC ARC.

    ↩︎