1
00:00:00,000 --> 00:00:02,677
How specifically does being open-source

2
00:00:02,760 --> 00:00:04,517
not lead us to our end?

3
00:00:04,600 --> 00:00:09,737
>> Dr. Yann LeCun, Turing Award Laureate,
Vice President and Chief AI Scientist

4
00:00:09,820 --> 00:00:13,037
[music] of Meta. >> We need
free and diverse AI platforms

5
00:00:13,120 --> 00:00:15,917
for the same reason that we need free
and diverse press, so we don't have a

6
00:00:16,000 --> 00:00:17,957
single opinion coming to us from AI.

7
00:00:18,040 --> 00:00:21,997
We're still very far from matching
human and animal [music] intelligence and

8
00:00:22,080 --> 00:00:23,957
learning capacities with AI systems.

9
00:00:24,040 --> 00:00:29,160
It's not just around the corner. It's
not going to happen in the next 2 years.

10
00:00:30,520 --> 00:00:32,837
>> Can you talk to us, Yann, about

11
00:00:32,920 --> 00:00:37,877
open-source AI? Why is open-source
the way forward from a security and

12
00:00:37,960 --> 00:00:40,997
innovation perspective? >> So,

13
00:00:41,080 --> 00:00:44,957
there's going to be a a future,
not very far in the future,

14
00:00:45,040 --> 00:00:48,517
where all of our interaction
with the digital

15
00:00:48,600 --> 00:00:52,317
world are going to be
mediated by AI systems.

16
00:00:52,400 --> 00:00:55,197
We already see this with uh

17
00:00:55,280 --> 00:00:57,277
uh companies putting out

18
00:00:57,360 --> 00:00:59,797
basically replacement for search engines

19
00:00:59,880 --> 00:01:03,077
uh that are backed by AI systems.

20
00:01:03,160 --> 00:01:07,557
And so, in this in this future,
if all of our digital diet is mediated by

21
00:01:07,640 --> 00:01:09,317
AI systems,

22
00:01:09,400 --> 00:01:11,877
um it's going to be there's going to be

23
00:01:11,960 --> 00:01:15,157
a need for a a a very wide diversity of

24
00:01:15,240 --> 00:01:19,077
AI systems that cater to different
languages, different cultures, different

25
00:01:19,160 --> 00:01:22,397
value systems,
different centers of interest.

26
00:01:22,480 --> 00:01:26,557
And it it can't be produced by a small
number of companies on the West Coast of

27
00:01:26,640 --> 00:01:27,837
the US.

28
00:01:27,920 --> 00:01:31,797
Um um or any single country.

29
00:01:31,880 --> 00:01:34,677
Uh so, essentially AI

30
00:01:34,760 --> 00:01:39,677
tools, assistants, will become kind
of a repository of all human knowledge.

31
00:01:39,760 --> 00:01:43,357
And um it's going to be a a kind of

32
00:01:43,440 --> 00:01:46,837
infrastructure. And the history shows that

33
00:01:46,920 --> 00:01:50,637
infrastructure software always
end ends up being open-source

34
00:01:50,720 --> 00:01:54,797
because it's more secure,
it's more customizable.

35
00:01:54,880 --> 00:01:58,317
Uh it enables the creation
of an ecosystem on top of it.

36
00:01:58,400 --> 00:02:02,877
And so the the entire infrastructure of
our communication systems, the internet,

37
00:02:02,960 --> 00:02:06,837
um even our cell phones, uh

38
00:02:06,920 --> 00:02:10,357
software stack for the, you know,
communication communicating with cell

39
00:02:10,440 --> 00:02:12,517
phone towers, it's all open source.

40
00:02:12,600 --> 00:02:15,877
For very good reasons. It's not
by design, it's really the the market

41
00:02:15,960 --> 00:02:19,757
forces that push it.
So there's a similar phenomenon which

42
00:02:19,840 --> 00:02:22,757
is definitely happening now in the last uh

43
00:02:22,840 --> 00:02:25,677
uh few months that Llama 2 has uh has

44
00:02:25,760 --> 00:02:29,117
been available and more open source uh

45
00:02:29,200 --> 00:02:32,757
uh and then it's been available
that people are using those open source

46
00:02:32,840 --> 00:02:37,037
models and and customizing them for
vertical applications in businesses, but

47
00:02:37,120 --> 00:02:41,677
also uh training them for,
you know, a particular types of uh

48
00:02:41,760 --> 00:02:42,317
uh

49
00:02:42,400 --> 00:02:45,237
languages or or cultures. Uh and so it's

50
00:02:45,320 --> 00:02:49,157
a necessity. We we need it. We need

51
00:02:49,240 --> 00:02:52,757
free and diverse AI platforms
for the same reason that we need free and

52
00:02:52,840 --> 00:02:55,797
diverse press. Um so we
don't have a single opinion

53
00:02:55,880 --> 00:02:58,197
coming to us from from AI. Getting

54
00:02:58,280 --> 00:03:00,717
towards the the title, will AI lead us

55
00:03:00,800 --> 00:03:05,437
to our end? How specifically
does being open source

56
00:03:05,520 --> 00:03:09,317
not lead us to our end? Is there
something fundamental that makes it

57
00:03:09,400 --> 00:03:12,077
easier to secure?

58
00:03:12,160 --> 00:03:16,397
Yes. Uh so it's it's certainly been
the case historically that uh open source

59
00:03:16,480 --> 00:03:20,237
software has been more
more secure simply because

60
00:03:20,320 --> 00:03:25,437
there's more eyeballs on it, right?
So the entire world today runs on Linux.

61
00:03:25,520 --> 00:03:28,117
Um maybe not your desktop,

62
00:03:28,200 --> 00:03:31,077
uh but probably more more than half of

63
00:03:31,160 --> 00:03:35,317
your phones run on Linux.
Whatever is not an iPhone runs on Linux.

64
00:03:35,400 --> 00:03:39,437
Uh the web servers around the world run
on Linux. All the cloud services run on

65
00:03:39,520 --> 00:03:43,357
Linux. Your car has several
microcontrollers that probably run

66
00:03:43,440 --> 00:03:46,557
Linux. The uh helicopter
on Mars runs Linux. I

67
00:03:46,640 --> 00:03:50,397
mean, Linux runs the world.
And the reason is because

68
00:03:50,480 --> 00:03:53,637
it's open source, so you can port it
to anything you want, and you can make it

69
00:03:53,720 --> 00:03:55,797
more secure. And and so, you know,

70
00:03:55,880 --> 00:03:58,397
there's kind of a similar story uh to be

71
00:03:58,480 --> 00:04:02,477
uh to be made for uh for for AI uh

72
00:04:02,560 --> 00:04:04,597
business models. More eyeballs, more

73
00:04:04,680 --> 00:04:11,080
ways to uh make it safe, um uh etc. Now,

74
00:04:11,440 --> 00:04:14,597
the only reason why you might think that

75
00:04:14,680 --> 00:04:18,837
it's not a good idea to open source
is if you have this belief that somehow AI

76
00:04:18,920 --> 00:04:22,357
is so powerful that it
is going to take over the world

77
00:04:22,440 --> 00:04:26,357
at some point. Okay. And uh

78
00:04:26,440 --> 00:04:29,757
and this is this is not happening.

79
00:04:29,840 --> 00:04:33,877
Um so, first of all, we're still very far

80
00:04:33,960 --> 00:04:37,077
despite all the wonderful
things that AI can do today,

81
00:04:37,160 --> 00:04:39,237
uh we're still very far from matching

82
00:04:39,320 --> 00:04:44,517
human and animal intelligence
and learning capacities with AI systems.

83
00:04:44,600 --> 00:04:47,397
We're very far. It's not
just around the corner. It's

84
00:04:47,480 --> 00:04:49,597
not going to happen in the next 2 years.

85
00:04:49,680 --> 00:04:52,397
Um we still some need some scientific

86
00:04:52,480 --> 00:04:55,117
breakthroughs to get to the type of

87
00:04:55,200 --> 00:04:57,637
learning abilities that we observe in uh

88
00:04:57,720 --> 00:05:00,597
not just humans, but even your your cat.

89
00:05:00,680 --> 00:05:03,957
Um your cat has a much better
understanding of the physical world than

90
00:05:04,040 --> 00:05:07,237
uh the most powerful
AI systems that we have.

91
00:05:07,320 --> 00:05:12,797
>> I've seen my dog run into the glass
door in my house. Y- Y- I I I I I debate

92
00:05:12,880 --> 00:05:15,077
that, but yeah. >> Sure,
but you know, I've run into glass

93
00:05:15,160 --> 00:05:19,917
doors. Um so, the the uh

94
00:05:20,000 --> 00:05:23,477
H- Here is an interesting uh statistic. Um

95
00:05:23,560 --> 00:05:27,277
so, first of all, before we get
to human-level AI, uh we're going to need

96
00:05:27,360 --> 00:05:30,717
some breakthroughs that allow machines
to understand how the world works, which

97
00:05:30,800 --> 00:05:35,437
they can't do at the moment,
uh can remember facts, which they can't

98
00:05:35,520 --> 00:05:38,677
really do at the moment. They don't
really have persistent memory, at least

99
00:05:38,760 --> 00:05:40,077
the LLMs don't.

100
00:05:40,160 --> 00:05:43,237
Um they can reason, and they can plan.

101
00:05:43,320 --> 00:05:48,277
And those are characteristics that are
essential to intelligent behavior.

102
00:05:48,360 --> 00:05:52,157
Current large language models cannot do
any of those things, at least not to the

103
00:05:52,240 --> 00:05:56,397
level that uh we'd like them to to do.

104
00:05:56,480 --> 00:05:58,557
And the reason is is relatively simple.

105
00:05:58,640 --> 00:06:00,877
Um if you

106
00:06:00,960 --> 00:06:05,637
if you look at how much data is used
to train those LLMs, it sounds staggering.

107
00:06:05,720 --> 00:06:09,557
It's uh typically 10 trillion
tokens, right? A token is

108
00:06:09,640 --> 00:06:14,637
like a word, so it's 10 to the 13,
right? A one with 13 zeros. Uh

109
00:06:14,720 --> 00:06:18,477
words or word equivalents.

110
00:06:18,560 --> 00:06:22,277
Each word or each of those tokens is
stored in two bytes, so that's 2 * 10 to

111
00:06:22,360 --> 00:06:26,597
the 13 bytes. Seems staggering.
If you evaluate how

112
00:06:26,680 --> 00:06:29,677
long it would take for humans to read
this, it would take, you know, anywhere

113
00:06:29,760 --> 00:06:34,637
between 150,000 and 200,000 years
for a single person to read this reading 8

114
00:06:34,720 --> 00:06:38,597
hours a day. Uh enormous. So,

115
00:06:38,680 --> 00:06:42,157
the conclusion is those systems are
incredibly smart, right? Because they've

116
00:06:42,240 --> 00:06:47,840
they're capable of digesting
this enormous amount of knowledge.