Prospecting for Gold | Owen Cotton-Barratt | EAGx 2016 Oxford

Prospecting for Gold | Owen Cotton-Barratt | EAGx 2016 Oxford

Now I’m going to introduce Owen Cotton-Barratt,
who is giving the keynote talk, which is labeled “Prospecting for gold: Techniques for finding
high-value opportunities”. Effective Altruism is an intellectual project trying to figure
out how to do the most good in the world, but how can you figure this out? In this talk,
Owen Cotton-Barratt of the Future of Humanity Institute and the Centre for Effective Altruism
walks us through a number of quantitative and qualitative models and discusses where
the EA movement should go next. Owen is a friend and colleague of mine at CEA and FHI,
is a mathematician currently working as a research fellow at FHI, and a senior research
adviser at CEA. Thank you, Michael. So I’m going to talk about this. The central metaphor that is going to be running
through my talk is ‘effective altruism as mining for gold’. And I’m going to keep
on coming back to this metaphor to illustrate different points. Gold, here, is a stand-in
for whatever it is that we truly value. Some things that we might value include making
more people happy and well-educated. Or trying to avert a lot of suffering. Or trying to
increase the probability that humanity makes it out to the stars. When you see gold, take
a moment to think about what you actually value. Many people won’t just value one
particular thing. However, do think about what you care about and put that in place
of the gold. And then there’s lots of observations we can make. This is a photo of Viktor Zhdanov, and I learned
about him by reading Will MacAskill’s book ‘Doing Good Better’. He was a Ukrainian
biologist, who was instrumentally extremely important in getting an eradication program
for smallpox to occur. As a result, he probably was counterfactually responsible for saving
tens of millions of lives. Obviously, we don’t all achieve this. So,
by looking at examples like this, we can notice that some people manage to get a lot more
gold, manage to achieve a lot more of whatever we altruistically value, than others. And
that is reason enough to make us ask questions like: what is it that gives some people better
opportunities than others? How can we go and find opportunities like that? Elsewhere in this conference, there are going
to be treasure maps and discussions of where the gold is. I’m not going to do that in
this talk. I’m instead going to be focusing on the tools and techniques that we can use
for locating gold, rather than trying to give my view of where it is directly. Another thing I want to cover here is, actually,
I’m using this metaphor. I want to say a little bit about why I’m
even using a metaphor – because we care about these things. We care about a lot of these
big, complicated, valuable things. Why would I try and reduce that down to gold? Well,
it is because of where I want the focus of this talk to be. I want the focus to be on
techniques, tools and approaches that we can use. And if you have complex values, that
we’re trying to put in the background, that actually would just keep on pulling your attention.
But a lot of the things that we might do to try to identify where valuable things are,
and how to go and achieve them, are constant, regardless of what the valuable thing is.
So, by replacing them with a super simple stand-in for value, I think it helps to put
the focus on this abstract layer that we are putting on top of that. In this talk, I’m going to go through a whole
lot of different ideas, and illustrate lots of them with this metaphor. Some of you will
be familiar with lots of all of these ideas already. If so, hopefully you can have fun
by kind of seeing a new metaphor. Maybe you can spend time thinking about exactly how
you could better the analogy or where I’ve gone wrong with the metaphor. If you haven’t
seen this stuff before, I’ll go reasonably fast but hopefully you can follow stuff. If
you do get lost on a point just don’t worry, it’s modular, like, try and pick it up when
we come to the next part. The first thing I’m going to talk about
is the fact that gold, like literal gold, is pretty unevenly spread around the world.
There are loads of places with almost no gold at all, and then there are a few places where
there’s a big seam of gold running into the ground. This has some implications. One
is that we would really like to find those seams. Another is about sampling. For some quantities,
if I want to know roughly how tall people are, sampling five people, measuring their
height and saying, “Well, the average is probably like that” is not a bad methodology.
However, if I want to know on average how much gold there is in the world, sampling
five random places, and measuring that, is not a great methodology, because it’s quite
likely that I’ll find five places where there’s no gold, and I’ll significantly
underestimate. Or possibly, one of them will have a load of gold, and now I’ll have a
massively inflated sense of how much gold there is in the world. This is a statistical property that loosely
gets called having a heavy tail on the distribution. This here is a distribution without a heavy-tail.
There is a range of different amounts of gold in different places, but none of them has
massively more or massively less than typical. Here, in contrast, is a heavy-tail distribution.
It looks similar-ish to the one on the left-hand side, but there’s this long tail, getting
up to very large amounts of gold, where the probabilities aren’t dying off very fast.
This has implications. Here is another way of looking at these distributions
(Figure 7). In this case, I’ve arranged, going from left to right, the places in order
of increasing amounts of how much gold they have. This is the percentiles, and then I’ve
just put the amount of gold on the vertical axis. In this case, I’ve colored in beneath
the graph. This is because that quantity – that area – is meaningful. It corresponds to actually,
the total amount of where that gold is. So, in this case, on the left, if the distribution
wasn’t heavy-tailed, I can see that the gold is fairly evenly spread across lots of different
places. If we want to just get most of the gold, what is important is getting to as many
different places as possible. Solar power is like this. Sure, some places
get more sunlight than other places, but the amount of solar power you generate depends
more on how many total solar panels you have, than on exactly where you place them. Over on the right, though, we have a distribution
where you can see a lot of the area is on that spike right at the right-hand side. This
just means that a lot of the gold, and if this is a true stand-in for something that
we value, a lot of what is valuable comes in this extreme of the distribution of things,
which are just unusually good. So at least literal gold, I think, is distributed
like this. Disclaimer, I’m not a geologist, I actually don’t know anything about gold,
but I understand that this is right. We might ask, is this also true of opportunities to
do good in the world? Here are a couple of bits of support for this. First there’s just general things, of when
we look into the world, and it’s pretty complex, we do see distributions with this
heavy-tail property coming up in a lot of different places. There are some theoretical
reasons to expect certain types of distribution to arise. And also, empirically, if we go
and look at something like income distributions around the world, again this is that percentile
version, and you can see the spike. Okay, that was for kind of in general, if
we just go and look at the world. Obviously, there are lots of things as well that don’t
have this property. But the more we look at things where there are complex systems with
lots of interactions, if often increases the degree to which we see this property. That
is a big feature of lots of ways that we try and interact to improve the world. I can also just try and look explicitly at
opportunities to do good. And I can see a couple of reasons why I personally am convinced
that we get some of this property. So, one reason is just convincing arguments. If I
care about stopping people starving, and I do care about stopping people starving, I
could ask: should I be interested in direct famine relief and trying to get food to people
who are starving today? I can compare this to something more speculative. I personally
have been convinced by the arguments in this book, that it would be more effective to focus
on doing research towards having solutions for feeding very large numbers of people in
the event that agriculture collapses. It’s pretty extreme. It’s not something we normally
think about, but I think that the argument basically checks out. This tells me that one
way of doing this, I’m just limiting myself here to trying to feed people, and one of
the mechanisms looks much more effective than the other. I can also look at data. This is data from
DCP2, which has tried to estimate the cost-effectiveness of lots of different developing world health
interventions. The x-axis is on a log scale, so these have been put into buckets, and each
column is on average ten times more effective to than the one to its left. Here the rightmost
column is about 10,000 times more effective than the leftmost column. And this was just
within one area where we have managed to get good enough data that we can go and estimate
these things. There is just a very wide range of cost effectivenesses. The implications of this are that if we want
to go and get gold, we really should focus on finding seams. In some cases, it might
give us this surprising conclusion, say you discover that something is at the 90th percentile,
that might make us less excited about it. Because before we knew anything, it might
have been anywhere on the distribution. And if most of the possible value of it comes
from it being up at the 99th percentile, then discovering it’s only at the 90th percentile
could actually be a bad thing. I mean, it’s a good thing to discover, but it makes us
think less well of it. Now, that’s if you’ve got a fairly extreme distribution, but it’s
interesting to see how you can get these kinds of counterintuitive properties. Another implication is that perhaps a kind
of naïve empiricism, “we’ll just do a load of stuff and see what comes out best”,
isn’t going to be enough for us in judging this, because of this sampling issue. We can’t
go and sample enough times and measure the outcomes well enough to judge how it’s actually
going to be. If we actually want to get as much gold as
possible, we want to go to a place where there is lots of gold. We want to have the right
tools for getting the gold out, and we want to have a great team that is going to be using
those tools. I think that we can port this analogy over to opportunities to doing good
as well. We can roughly measure the effectiveness of the area or type of thing that we’re
doing, and the effectiveness of the intervention that we’re doing to create value in that
area, relative to other interventions in the area. And the effectiveness of the team or
the organisation who is implementing that, relative to how well other teams might implement
such an intervention. And if you have these things then the total
value that you are going to be getting is equal to the product of these. I’ve represented
it here by volume, and we want to be maximising the volume. That means we’re going to want
to be trying to do fairly well on each of the different dimensions. At least, not terribly
on any of the dimensions. Some implications there might be that if we have an area and
an intervention that we’re really excited about, but we can only find a kind of bad,
mediocre team working on it, it may be better not to just support them to do more of that,
but to try and get somebody else working on it. Or to do something to really improve that
team. Similarly, we might not want to support even a great team, if they’re working in
an area that doesn’t seem important. In the next part, I am going to talk about
the tools and techniques for identifying where in the world gold is. A nice property of literal
gold is that when you dig it up, you’re pretty sure that you can recognise “yes,
I have gold”. We often have to deal with cases where we don’t have this. We don’t
have the gold, so we have to carefully try to infer its existence, by using different
tools. This fact is like the dark matter of value. So that increases the importance of having
good tools, for trying and actually measure and assess this. And it increases the importance
of actually applying those tools diligently as well. Actually, that was iron pyrite, not
gold. That’s more what gold looks like. The point there is, just because somebody says,
“Hey, this is gold,” it doesn’t mean we should always take people’s word on this.
It does provide some evidence, but… It gives a motivation for wanting to have great tools
for identifying particularly valuable opportunities. And being able to differentiate and say, “Okay,
actually this thing, although it has some aspects of value, may not be what we want
to pursue.” If you first go to an area where nobody has
been before, then the seams of gold that are running through the ground have often been
eroded a little bit, and you can have little nuggets of gold just lying around on the ground,
and it’s extremely easy to get gold. So you have some people go in, they do this for
a bit, and they run out of all the gold on the ground. And now, if they want to get more gold, maybe
more people come along, they bring some shovels, and it is a bit more work, but you can still
get gold out (figure 15). And then you dig deep enough, and you can’t
just get in with shovels anymore, so you need bigger teams and heavier machinery to get
gold out. You can still get gold, but it’s more work for each little bit, for each nugget
that you’re getting out. This is the general phenomenon of ‘diminishing returns’ on
work that you’re putting in. I think that this actually comes up in a lot of different
places, and so it is worth having an idea about it. By the way, this, like several of the things
I’m going to be talking about, is a concept, which, I guess, is native to economics. And
in some cases, I’m merely just pulling this from economics, and in some cases, there’s
a little bit more modification on the concept. For instance, I think that we get this in
global health. I understand that 15 or 20 years ago, mass vaccinations were extremely
cost-effective and probably the best thing to be doing. Then the Gates Foundation has
come, in and they funded a lot of the mass vaccination interventions. Now, the most cost-effective
intervention is less cost-effective than mass vaccinations. And that’s great, because we
have taken those low hanging fruit. Or similarly, if in AI safety, writing the first book on
superintelligence is a pretty big deal. Writing the 101st book on superintelligence is just
not going to matter as much. So, a minute ago, I talked about how we could
factor the effectiveness of organisations into the area in which they were working,
the intervention they were pursuing, and the team working on it. Now, I’m going to focus
on that first one, trying to assess the area. And I’m going to give a further factorization,
splitting that into three different things. The first of these dimensions is scale. All
else being equal, we would prefer to go somewhere where there is a lot of gold, rather than
a little bit of gold. And probably per unit of effort, we are going to get more gold,
if we do that. Second: tractability. We’d like to go somewhere
where we make more progress per unit of work. Somewhere, where it’s nice and easy to dig
the ground, rather than trying to get your gold out of a swamp. And third is uncrowdedness. This has sometimes
been called neglectedness. I think that term is a bit confusing. It’s a bit ambiguous
because sometimes people use ‘neglectedness’ to just mean, all things considered, that
this is an area which we should allocate more resources to. What I mean here is just that
there aren’t many people looking at it. All else being equal, we would rather go to
an area where people haven’t already picked up the nuggets of gold on the ground, than
one where they have, and now the only gold remaining is quite hard to extract. Ideally, of course, we’d like to be in the
world where there is loads of gold, that’s easier to get out, and nobody has taken any
of it. But, we are rarely going to be in that exactly ideal circumstance. So one question
is: how can we trade these off against each other? I’m going to present one attempted
way to try and make that precise. I’ve allowed myself one equation in this talk. This is
it. If you’re not used to thinking in terms
of derivatives, just ignore the ‘ds’ here. But this thing is the value of a little bit
of extra work, so this is generally what we care about if we’re trying to assess which
of these different areas should we go and do more work on. And then here’s a factorization. This is mathematically
trivial. I’ve just taken this one expression, and I’ve added in a load more garbage. And
on the face of it, it looks like I’ve made things a lot worse. I can only justify this,
if it turns out that these terms I’ve added in, which cancel each other out, actually
mean that the right-hand side here is easier to interpret or easier to measure. So I’m
going to present a little bit of a case for why I think it is. This first term here is measuring the amount
of value you get for, say, solving an extra one percent of a solution. And that roughly
tracks how much of a big deal the whole problem that you’re looking at is, the whole area.
I think that is a pretty good, precise version of the notion of scale. The second one is a little bit more complicated.
It is an elasticity here, which is a technical term. It’s a pretty useful and general term
(go look it up on Wikipedia, if you’re interested). But, it’s measuring here, how much does a
proportional increase in the amount of work that’s being done, leads to… what proportion
of a solution does that give you. People have talked about this kind of scale,
tractability, uncrowdedness framework for a few years without having a precise version.
That means that people have given different characterizations of the different terms,
and I think there have been a few different versions of tractability. Not all of them
line up with this exactly. But I think that this idea of it – measuring how much more
work gets you towards a solution – is fairly well captured by this version here. And this final thing actually just cancels
to one over the total amount of work being done. So that is very naturally a measure
of uncrowdedness. I think that all of these dimensions matter.
And again, that means we probably don’t want to work on something, which does absolutely
terribly on any of the dimensions. I’m not going to spend an hour helping a bee, even
if nobody else is helping it and it would be pretty easy to help, because just the scale
of it is pretty small. I don’t think we should work on perpetual motion machines,
even though basically nobody is working on it and it would be really fantastic if we
succeeded. Because it seems like it’s not tractable. And there’s nothing which is extremely crowded,
just because there’s only 7 billion people in the world, so you can’t force this that
low. But this might give us a warning against actually
working on climate change. Because at a global scale, that gets a lot of attention, as a
problem. I’m going to add some more caveats to that
one. One is that this is going to be true while we think that there are other problems,
which are just significantly more under-resourced. And another is that you might think that you
have an exception if you have a much better way of making progress on the problem of climate
change than typical work that is being done on it. Even so, I think maybe we should think it’s
a bit surprising that I’m making a statement like “climate change is not a high priority
area.” This just sounds controversial and we should be skeptical of this. But, I think
that the term “high priority” is a little bit overloaded. And so I want to distinguish
that a little bit. If we have these two places where there’s
gold in the ground, and we say: “Where should we send people if we want to get gold?”
The answer is going to depend. Maybe we send the first person to this place on the right,
where there’s only a little bit of gold, but it’s really easy to get out. Then we
send the next ten people to the place on the left, just because there’s more total gold
there. The first person will already have gotten most of the gold on the right. And
we want more people total working on this place on the left. Which of these is higher
priority? Well, that just depends on which question you’re actually asking. These numbers are just made up off the top
of my head, but we might have some distribution like this. Here, we ask the question “How
much should the world spend on this area, total?” and we get one distribution, where
maybe climate change looks very big on this. Can people read that text? Raise your hand
in the back if you can’t read the text. Great. And if we instead ask, how valuable is marginal
spending? The graphs actually might look quite different because here it’s a function of
several things, but significantly how much is already being spent. I’ve added some dotted
black lines on the diagram on the left – they might represent how much is already being
spent. Then, the graph on the right is a function of all sorts of things, like
how much should be spent in total, how much is already being spent and of course, what
the marginal returns are – what the curve looks like there. But I think that both of these are actually
important notions, and which one we use should depend on what we are talking about. If we
are having a conversation about what we as individuals or as small groups should do,
I think it’s appropriate to use this notion of marginal priority, of how much do extra
resources help. If we’re talking about what we collectively as a society or the world
should do, I think it’s often correct to talk about this kind of notion of absolute
priority and how much resources ought to be invested in it, total. Okay, this is a – for most of the things here,
I’ve been extremely agnostic about what our view of value is. Just for this point,
I’m going to start making more assumptions. I think quite a few people have the view that
what we want to do is try and make as much value over the long term, as we can. Some
people don’t have that view. Some people haven’t thought about it. If you don’t
have that view, you can just treat this as a hypothetical: “Now I can understand what
people with that view would think.” If you haven’t thought about it, go away and think
about it, some time. It’s a pretty interesting question, and I think it’s an important
question, and is worth spending some time on. But, if we do care about creating as much
value in the long term as possible, in our gold metaphor, that might mean wanting to
get as much gold out of the ground eventually as possible, rather than just trying to get
as much gold out of the ground this year. And maybe we have some technologies which
are destructive. So we can use dynamite and dynamite gets us loads of gold now, but it
also blows up some gold, and now we never get that gold for later. That could be pretty
good if you are focusing just on trying to get gold in the short term. But it could be
bad from this eventual gold perspective. If we have different technologies that we
can develop, maybe we can develop some that are also efficient but less destructive. And
there are going to be some people in the world who do care about creating as much gold as
possible in the short term. They are going to use whichever technology is the most efficient
for that. And so one of the major drivers of how much gold is eventually extracted is
the order in which the technologies are developed, and the sequencing. If we discover the dynamite
first, people are going to go and have fun with their dynamite and they’re going to
destroy a lot of the gold. If we discover the drill first, then by the time dynamite
comes along, people will go “Well, why would we use that? We have this fantastic drill.” Philosophers like Nick Bostrom have used this
to argue for trying to develop societal wisdom and good institutions for decision making,
before developing technologies or progress which might threaten the long-run trajectory
of civilization. And also for trying to focus on differentially aiming to develop technologies
which enhance the safety of new developments, rather than or before anything that’s driving
risk. So in part three, I’m going to talk about
how this is a collaborative endeavour. We’re not just all, each of us individually, going:
“I need to work out where the most gold is. And that’s most neglected, most tractable.
I personally am just going to go and do that.” Because there is a whole lot of people who
are thinking like this, and there are more every year. I am really excited about this.
I’m really excited to have so many people here and also this idea that maybe in two
years time, we’ll have a lot more again. But, then, we need to work out how to cooperate.
Largely, we have the same view or pretty similar views over what to value. Maybe some people
think that silver matters, too – it’s not just gold – but we all agree that gold matters.
We’re basically cooperating, here. We want to be able to coordinate and make sure that
we’re getting people working on the things which make the most sense for them. There is this idea of comparative advantage.
I have Harry, Hermione and Ron, and they have three tasks that they need to do in order
to get some gold. They need to do some research, they need to mix some potions, and they need
to do some wand work. Hermione is the best at everything, but she doesn’t have a time
turner, so she can’t do everything. So we need to have some way of distributing these.
This is the idea of comparative advantage. Hermione has an absolute advantage on all
of these tasks, but it would be a waste for her to go and work on the potions, because
Harry is not so bad at potions. And really, nobody else is at all good at doing the research
in the library. So we should probably put her on this. And this is a tool that we can use to help
guide our thinking about what we should do as individuals. If I think that some technical
domain and technical work is the most valuable thing to be doing, but I would be pretty mediocre
at that, and I’m a great communicator? Then maybe I should go into trying to help technical
researchers in that domain communicate their work, in order to get more people engaged
with it and bring in more fantastic people. So that’s applying this at an individual level.
We can also apply this at the group level. We can notice that different organizations
or groups may be better placed to take different opportunities. This is a bit more speculative, but I think
we can also apply this at the time level. We can ask ourselves, “What are we, today,
the people of 2016, particularly well suited to do, versus people in the past and people
in the future?” We can’t change what people in the past did. But we can make this comparison
of what our comparative advantage is relative to people in the future. And if there is a
challenge, if there were going to be some different possible challenges in the future
that we need to meet, it makes sense that we should be working on the early ones. Because
if there’s a challenge coming in 2020, the people in 2025 just don’t have a chance to
work on that. Another thing which might come here is that
we have a position, perhaps, to influence how many future people there will be who are
interested in and working on these challenges. We have more influence over that than people
in that future scenario do, so should we think about whether that makes sense as a thing
for us to focus on. Another particularly important question is
how to work stuff out. The world is big and complicated and messy. And we can’t expect
all of us, individually, to work out perfect models of it. In fact, it is too complicated
for us to expect anybody to do this. So, maybe we’re all walking around with the little
ideas which, in my metaphor here, are puzzle pieces for a map to where the gold is. We
want institutions for assembling these into a map. It’s a bit complicated, because some
people have puzzle pieces which are from the wrong puzzle, and doesn’t actually track where
gold is. Ideally, we would like our institutions to filter these out and only assemble the
correct pieces to guide us where we want to go. As a society, we have had to deal with this
problem in a number of different domains, and we have developed a number of different
institutions for doing this. There is the peer review process in science. Wikipedia
does quite a lot of work aggregating knowledge. Amazon reviews aggregate knowledge that individuals
have about which products are good. Democracy lets us aggregate preferences over many different
people to try and choose what’s actually going to be good. Of course, none of these institutions is perfect.
And this is a challenge. This is like one of those wrong puzzle pieces, which made it
into the dialogue. And this comes up in the other cases as well. The replication crisis
in parts of psychology has been making headlines recently. Wikipedia, we all know, sometimes
gets vandalized, and you just read something which is nonsense. Amazon reviews have problems
with people making fake reviews, to make their product look good or other people’s products
look bad. So, maybe it is the case that we can adapt
one of these existing institutions for our purpose, which is trying to aggregate knowledge
about what are the ways to go and do the most good. But maybe we want something a bit different,
and maybe somebody in this room is going to do some work on coming up with valuable institutions
for this. I actually think this is a really important problem. And it’s one that is
going to become more important for us to deal with as a community, as the community grows. That was all about what are our global institutions
for pulling this information together and aggregating it. Another thing, which can help
us to move towards getting a better picture, is trying to have good local norms. So, we
tell people the ideas that we have, and then other people maybe start listening. And sometimes
it might just be that they listen based on the charisma of the person who is talking,
more than based on the truthiness of the puzzle piece. But we’d like it to be – to have ways
of promoting the spread of good ideas, inhibiting the spread of bad ideas, and also encouraging
original contributions. One way of trying to promote the spread of good ideas and inhibit
bad ideas is just to rely on authority. We’ll say, “Well, we’ve worked out this stuff.
We’re totally confident about this. And now we just won’t accept anything else.”
But that isn’t going to let us actually get new stuff. I think something to do here is to pay attention
to why you believe something. Do you believe it because somebody else told you? Do you
believe it because you have really thought this through carefully and worked it out for
yourself? There is a blur between those. Often somebody tells you, and they kind of give
you some reasons. And you are thinking: “Oh, those reasons kind of check out,” but you
haven’t gone and deeply examined the argument yourself. I think it’s useful, to be honest with yourself
about that. And then also to communicate it to other people. To let them know why it is.
Is it the case that you believe this because Joe Bloggs told you? And actually, Joe is
a pretty careful guy, and he is pretty diligent about checking out his stuff, so you think
it probably makes sense. You can just communicate that. Or is it that you cut out this puzzle
piece yourself? Now, cutting it out yourself doesn’t necessarily
mean we should have higher credence in it. I have definitely worked things out, and I
have thought I have proved things before, and there was a mistake in my proof. So you
can separately keep track of the level of credence you have in a thing, and why you
believe it. Also, our individual and collective reasons
for believing things can differ. Here is this statement, that it costs about $3,500 to save
a life from malaria. I think this is broadly believed across the effective altruism community.
I think that collectively, the reason we believe this is that there have been a number of randomized
controlled trials. And then some pretty smart, reasonable analysts at GiveWell have looked
carefully at this, and they have dived into all the counterfactuals, and they have produced
their analysis, and come to the conclusion: “On net, it looks like it’s about $3,500.” But that isn’t why I believe it. I believe
it because people have told me that the GiveWell people have done this analysis and they say
it’s $3,500. And they say, “Oh, yeah. I read it on the website.” That was why
I believed it until I started prepping for this talk when I went and read it on the website.
Because I think that this is a bit more work for me, but it’s doing a bit of value for
the community. I’m shortening the chain of passing this message along. And as things
get passed along, it’s more possible that mistakes enter or just something isn’t well
grounded, and then it gets repeated. By going back and checking earlier sources in the chain,
we can try to reduce that, and try to make ourselves more robustly confident in these
statements. Another thing that comes up is when you notice
that you disagree with somebody: If you’re sitting down and talking with someone and
they’re saying something, and you think: “Well, that’s obviously false.” You
can see perhaps that parts of their jigsaw puzzle are wrong. You could just dismiss what
they have to say. But I think that that’s often not the most productive thing to do.
Because even if parts of what they have to say are wrong, maybe they have some other
part that is going into their thinking process which would fill a gap in your perspective
on it, and help you to have a better picture of what’s going on. I often do this when I find that someone has
a perspective that I think is unlikely to be correct. I’m interested in this process
of how they get there and how they think about it. Partly this is just that people are fascinating
and the way that people think is fascinating, so this is interesting. But I also think that
it is polite and I think it is useful. I think it does help me to build a deeper picture
of all the different bits of evidence that we have collectively. In this section, I’m going to put the stuff
I’ve just been talking about into action. I’ve told you about a whole load of different
things through this talk. But I didn’t tell you much about exactly what my level of confidence
in these is, or why I believe these. So, I’m going to do that here. I’m aware that nobody ever goes away from
a talk saying, “Oh, that was so inspiring. The way she carefully hedged all her statements.”
But I think it’s important. I would like people to go away from talks saying that.
So I’m just going to do it. Heavy-tailed distributions: I think it’s
actually pretty robust that the kind of baseline distribution of opportunities in the world
does follow something like this, a distribution with this heavy-tailed property. I think that
just seeing this in many different domains and understanding some of the theory behind
why it should arise makes it extremely likely. I think that there’s an open empirical question
to exactly how far that tail goes out. Heavy-tailedness isn’t just a binary property; it’s a continuum.
Anders Sandberg is going to be talking more about this, I think, later today. But, there is an important caveat here. This
is the only one of these where I’ve allowed myself a digression. It is that there is a
mechanism which might push against that, which is people seeking out and taking the best
opportunities. If people are pretty good at identifying the best opportunities, and they
are uniformly seeking out and taking them, then the best things that are left might not
be so much better. And this comes up in just regular markets.
Ways to make money, maybe they actually start out distributed across a wide range. This
is a log scale now, and it is meant to represent one of those heavy-tailed distributions, but
then people who are losing money, say, “Well, this sucks,” and they stopped doing that
thing. And they see other people who are doing activities which are making lots of money.
And they think: “Yeah, I’m going to go do that.” And then you get more people going
into that area, and then diminishing returns mean that you actually make less money than
you used to, by doing stuff in that area. So, afterwards, you end up with a much more
narrow distribution of the value that is being produced by people doing these different things,
than we started with. We might get a push in that direction among
opportunities to create altruistic value. I certainly don’t think that we are at a
properly efficient market. I’m not sure how efficient it is, how much we are curving
that tail. I hope that as this community grows, as we get more people who are actively trying
to choose very valuable things, that will mean the distribution does get less heavy-tailed,
because of this. One of the mechanisms that lead to efficiency
in regular markets is feedback loops, where people just notice they are getting rich or
that they are losing money. Another mechanism is people doing analysis, and they do this
because of the feedback loops, trying to work out that actually, we should put more resources
there because then we’ll get richer. I think that doing that analysis is an important part
of this project we’re collectively embarking on here. So, overall, I don’t think that we do have
an efficient market for this. I do think we have heavy-tailed distributions. I’m not
sure how extreme, but that’s because of that fact that it responds to actions people
are taking. Factoring cost-effectiveness: I think that
basically, this is just an extremely simple point, and there isn’t really space for
it to be wrong. But there is an empirical question as to how much these different dimensions
matter. It might be that you just have way more variation in one of the dimensions than
others. Actually, I don’t have that much of a view over how much the different dimensions
matter. We saw that the intervention effectiveness within global health varied by three or four
orders of magnitude. Area effectiveness, I think, maybe can be more than that, but I’m
not sure how much more. In terms of organization effectiveness, I’m just not an expert, and
I don’t want to try and claim to have much of a view on that. Diminishing returns: I just think this is
an extremely robust point. Sometimes, in some domains, there are actually increasing returns
to scale, where you get efficiencies of scale, and that helps you. I think that more often
applies at the organization scale or organization within a domain. Whereas diminishing returns
often apply at the domain scale. But I do know some smart people who think that I am
overstating the case for diminishing returns. So although I think, personally, that there’s
a pretty robust case, I would add a note of caution there. Scale, tractability, neglectedness: I think
it is obvious that they all matter. I think it is obvious, it is just trivial, that this
factorization is correct as a factorization. What is less clear is whether this breaks
it up into things that are easier to measure and whether this is a helpful way of doing
it. I think it probably is. We get some evidence from the fact that it loosely matches up with
an informal framework that people have been using for a few years, and have seemed to
find helpful. Absolute and marginal priority: Again, the
point, at some level, is just trivial. I made this point about communication because I think
not everybody has these separate notions and we can confuse each other if we blur them. Differential progress: I think that this argument
basically checks out. It appears in a few academic papers. It’s also believed by some
of the smartest and most reasonable people I know, which gives me some evidence that
it might be true, outside from my introspection. It hasn’t had that much scrutiny, and it’s
a bit counterintuitive, so maybe we want to expose it to more scrutiny. Comparative advantage is just a pretty standard
idea from economics. Normally markets try to work to push people into working in the
way that utilizes their comparative advantage. We don’t necessarily have that when we’re
aiming for more altruistic value. The application across time is also a bit
more speculative. I’m one of the main people who has been trying to reason this way. I
haven’t had anybody push back on it, but take it with a bit more salt, because it’s
just less well checked out. Aggregating knowledge: I think everyone tends
to think that yes, we want intuitions for this. And I think there is also pretty broad
consensus that the existing institutions are not perfect. Whether we can build better institutions,
I’m less certain. Stating reasons for beliefs: this again is
something that I think it’s common sense that all else equal, this is a good thing. But
of course, there are costs to doing it. It slows down our communication. And it may just
not sound glamorous and therefore be harder to get people on board with this. I think
that at least we want to nudge people in this direction, but I don’t know exactly how
far in this direction. We don’t want to be overwhelmingly demanding on this. I also
again, to some extent, believe this because a load of smart, reasonable people I know,
think that we want to go in this direction. And I weigh other people’s opinions quite
a lot when I don’t see a reason that I should have a particularly better perspective on
it than them. Finally, why have I been sharing all of this
with you? You know, people can go and mine gold without understanding all these theoretical
things about the distribution of gold in the world. But, because it’s invisible, we need
to be more careful about aiming at the right things. And so I think it’s more important
for our community to have this knowledge broadly spread. And I think that we are still in the
early days of the community and so it’s particularly important to try and get this
knowledge in at the foundations and to work out better versions of this. We don’t want
to have the kind of gold rush phenomenon where people charge off after a thing, and it turns
out there wasn’t actually that much value there. Finally, I’ll just point out too, if you like
this stuff, places where you can go and find out more about a lot of this. So, the whole
prioritisation track at the conference is I think mostly about discussing where to find
gold, which is one level more applied than this. There are a couple of talk duos which
I think are mostly about finding techniques and tools for doing this. And then, some of
the stuff on the community track, particularly these workshops on navigating intellectual
disagreement, I think are a pretty close fit for some of the things I was talking about
near the end. There’s loads of things I could point people
to for reading. I just chose a few here. If you want the slides for this talk, you can
find them online. One thing at the end here is I just said “economics”. Like, seriously,
it’s a big discipline. They’ve worked lots of stuff out. I’m sure that there’s a lot
of stuff out there which I would think is super relevant and I know quite a bit of economics,
but, it’s pretty interesting. And then, a couple of other opportunities.
One is that there is a student group launching here at Oxford next term, open to non-students
as well. We’re going to be meeting through the term, to decide how to allocate 10,000
pounds towards achieving as much good as we can. Another is that the Centre for Effective
Altruism is taking applications for Summer Research Fellowships. So you can find that
online. I’ll wrap up there. Thanks, everybody.


  1. Do you want to find the "Gold" that the human swarm values… here is the tool to use!

  2. Also watch Owen Cotton-Barratt's talk on AI at EAG 2017 San Francisco:

Leave a Reply

Your email address will not be published.