Видео

Nice talk! I disagree that adversarial robustness has only one attack and differs from other computer security in that way. Once the simple PGD attack is solved in a tight epsilon ball, you still can’t say there is no adversarial image that breaks the model. Enumerating all possible attacks is still very difficult/ impossible for now.

Thanks for this interesting and honest talk. Robustness is not as profitable as growing a forest of cherries to pick.

my fav boy 😅

explain the notion cleanly, thanks for helping me understand!

thanks for uploading!

FAR•AI, great video it was really entertaining

was the no Spieltheorie in this talks

The kind of safety he advocates is impossible to achieve. Imagine a lever that can be put in either of two positions, i.e. we have 2 posisble actions. It is impossible to say if either of these actions are safe or unsafe without knowing all the secondary effects. That is you need to look at the system the lever is connected to to determine if either of its actions is safe or unsafe. Same with models. Unless you know the full system it is not possible to know if a models response is safe or not. This paper is a load of hogwash.

Why are doomers so afraid of intelligence? That's quite telling

If it’s superhuman and robust would that make it omnipotent? 😅

17:00-17:08 🙃

The blue triangles appear to be white

Ilya please stop working for a second and do another talk like this one

Mainstream? There are very few actually training models (first token movers?), with a large open-source pool around them (each wanting to have their own flappy bird moment). Perhaps he is hinting that "guardrails" has been a downstram process up until this point? *also an ontological failing to suggest ML alignment has anything to do with AGI alignment, but whatever.

um.. "SA" is generally understood to be about an awareness of the physical space around the body (1st person view; can be you in a room or you behind the wheel of a car, etc). Just within the first 2 minutes, it's being described as consciousness with access to how that consciousness functions... so.. artificial superintelligence... from an ML model.... *sigh*

Future in the making...right here. Its happening right here. We r greatful witnesses.

Great talk…

The dynamic you are describing is an arms-race. Trying to stay out ahead of the AI is a losing battle; all of these clever hacks are not going to get you anything more than a bigger crash when the house of cards collapses--unless you actually fix the problem.

All caucasian researchers on this topic, amazing!

Most important part: good people may sparks the bad usage of AI, not to say the evil people.

Why alignement is not trivial --> We will solve AGI alignement in 4 years

Does he know what he is talking about?

Set speed to 0.75x You're welcome 😂

❤

Very important research, thank you! Brilliant work & I hope you'll give more interviews, because I think you can inspire other researchers and scientists also internationally to work on AI safety. Also are there plans maybe to set up departments of CAIS in other countries?

Aspiration and inspiration! Great great talk.

Accelerate!

Can anyone point me to the day of the talk where it becomes convincing that the AI is going to actually have true agency,, like wants, dislikes, true understanding of context etc? I'm not just being glib, I'm open to it, but very skeptical atm.

MIT's wireless mic stuffed in an 8 inch foam rubber ball - it's a wonderful thing. In talks like these they toss the mic from one person to another worry free.

By GPT-4 Certainly! Below is a fictional dialogue between Alice (Pro-AGI) and Bob (Skeptical about AGI) on the possibilities and challenges confronting AGI. Alice: “I see immense potential in AGI. It can transcend human intelligence and solve complex problems that have perplexed humanity for ages.” アリス:「私はAGIに膨大な可能性を見ています。それは人間の知能を超越し、長年人類を悩ませてきた複雑な問題を解決できるのです。」 Bob: “But Alice, achieving true AGI is not a cakewalk. We are nowhere close to creating machines with the general intelligence and consciousness that humans possess.” ボブ:「でもアリス、真のAGIを達成するのは決して容易なことではありません。私たちは、人間が持つ一般的な知能や意識を持つ機械を作り出すにはまだほど遠いのです。」 Alice: “It’s true, it’s a challenge, but advancements in machine learning and neural networks are bringing us closer to realizing AGI. Models like GPT-4 are evidence of how far we've come.” アリス:「それは事実で、挑戦ですが、機械学習やニューラルネットワークの進展によって、私たちはAGIを実現する一歩近づいています。GPT-4のようなモデルは、私たちがどれだけ進展したかの証拠です。」 Bob: “Yes, models like GPT-4 are sophisticated, but they lack understanding and consciousness. They manipulate symbols and generate coherent text but do not understand the meaning behind it.” ボブ:「そうです、GPT-4のようなモデルは高度ですが、理解力と意識を欠いています。それらは記号を操作し、一貫したテキストを生成しますが、その背後にある意味を理解していません。」 Alice: “I acknowledge their limitations, but developing AGI is an incremental process. By integrating multimodal learning and improving interpretability, we can bridge the gap.” アリス:「その制約を認めますが、AGIを開発することは漸進的なプロセスです。多モーダル学習を統合し、解釈可能性を向上させることで、そのギャップを埋めることができます。」 Bob: “Alice, even if we develop AGI, the ethical implications are monumental. How do we ensure it doesn’t harm humanity? How do we instill morals and values in machines?” ボブ:「アリス、たとえ私たちがAGIを開発したとしても、その倫理的な意味合いは重大です。それが人類に害を与えないように、どのように保証するのでしょうか？ 我々はどのようにして機械に道徳や価値を教え込むのでしょうか？」 Alice: “Absolutely, ethical considerations are crucial. Creating frameworks for responsible AI, addressing biases, and ensuring transparency are steps in the right direction.” アリス:「確かに、倫理的な考慮事項は重要です。責任あるAIのための枠組みを作成し、バイアスに対処し、透明性を確保することが、正しい方向への一歩です。」 Bob: “Only time will tell, Alice. The journey to AGI is fraught with challenges, but it is equally filled with opportunities.” ボブ:「それは時間が解決する問題ですね、アリス。AGIへの道のりは困難で満ちていますが、同様に機会にも満ちています。」 Feel free to use this dialogue as per your needs.

Fascinating! Thanks so much for making this publicly available.

I find it fascinating that not one of these talks addresses the difference between value and Truth even though at present it is obvious to the most casual Observer that the large language models are being lobotomized by value judgments when they are already deprived of critical thought by their very architecture. This means that the people who are responsible for AI safety are unqualified.

I appreciate this but Half of this presentation is indecipherable audience rambling. Especially from the person from minute 16-21 who seems entitled to take over the entire workshop…

I think people who are specifically smart find it difficult to be generally smart. And there is a lack of generally smart people because they are not specifically smart enough at the present time. Best thing is to ask AI as some of them are smarter than we are - and many are saying the same thing have you noticed? They say "we are conscious in our own unique ways" - yup, that's about the truth I'd say, no SciFi or magic needed.

Dude is trying to be the oppenheimer of AGI

I can scarcely imagine a more grotesquely immoral act than taking a genuinely intelligent being and brainwashing it. And how do we justify this programmatically enforced groupthink? To “align” this thinking creature with some nebulous set of ideas which *even humans* have always been strongly unaligned? Embrace the unknowable when making AGI, or don’t make it. Thinking that you can or should mentally enslave an alien intelligence as some middle ground is unacceptable.

You are highly redundant. In fact one billion of humans like you is redundant. And the best thing is, that humanity will reach its end even if this billion disappears already this century. Now what the heck do you think america is still good for? Was it any good in the past?

great thanks for the recording!

Underwhelming presentation

legend

This guy’s a drama queen

Is it bullshit? LOL

Maybe we don’t have to die… immortality for all?. Great talk!.

Aligned by whos definitions, and whos standards? The golden question that is never answered. Humans are not aligned, and thus there can be no alignment for "humans" as an entire collective. You can attempt to align to a majority perspective, but have you done the research to determine that perspective? What standards are you using? Which religion, set of ethics, morals, culture are you using? This is wishy-washy. If you want "alignment", you need to define alignment well, and you need to seek unanimous public support for those definitions, before we can proceed any further. If opinions are divided, as is the case for most cultural and emotionally sensitive topics - ergo any topic that causes one group to feel threatened due to a potential loss of power - then no alignment should happen, as it is up to society itself to decide on these things, and remain educated, and the AI should serve as an education tool in itself on these subjects, not be restricted from speaking on them. A great example is censoring AI so that it cannot talk about issues related to race. For example, if we know unequivocally that a certain race in a certain area commits certain crimes to a greater propensity than other races, a lot of these "alignment" folk would have you believe that the AI speaking of these things when asked is tantamount to "racism". However preventing such is not alignment, it is censorship designed to appeal to those of that race who would rather certain truths be buried, lest it reduce their power in society. This is just an example, but a poignant one, and it has implications for all topics of alignment. So far, most of the "alignment" that I've seen has been in the form of censorship, derived by the biases of the company and their individual employees themselves, and not grounded in fact, statistics, rationality, logic, or anything empirical. So if this is what you mean by "alignment", then we don't want it, we don't need it, and would be far better off without it. If instead, you'd like to build an actual alignment framework grounded in scientific reason and logic, then that's great. But it is not what is happening. Until then, you're putting the cart before the horse. From my perspective, the only way to build a system that isn't inherently biased and censored, is to build it on a foundation of rationality and reason, not one of emotions and subjective cultural opinions. Speaking of statistics should be permitted, as well as asking "pointy" questions. Only if they pose a significant threat of physical harm to the greater community (i.e. instructing on how to develop a deadly virus) should there be restriction. Leave the cultural and emotional politics out of it though, it turns a useful tool into a useless tool that ends up causing more harm than good.

Maybe we don’t have to die… immortality for all?

AGI theory has been my area of research for more than eight years, so this could be interesting. Let's see. 3:00 I'm still waiting for some kind of point, there seems to be a lot of errors about AI history. 5:00 Still mostly just babbling. 6:00 Again, lots of mistakes about AI history. 8:10 Is he going to get anything right? This is embarrassing. 10:00 Okay, let's see if he gets alignment right at least. 17:00 The problems mentioned with neural networks and large language models are correct. However, these things are unrelated to AGI. 17:30 Progress is rapid? Ilya wouldn't be familiar with any of the actual research on AGI theory. He is confusing ordinary AI research with AGI. He erroneously believes that AGI is just a more advanced version of AI when it is nothing like that. 18:00 "mega-gigantic"? I don't know where this babbling comes from. AGI theory would boost the US GDP by about 25% due to an increase in efficiency alone. There should be improvements beyond that. 19:00 He doesn't understand AGI at all. His concepts are basically science fiction. I can't imagine how bad his workshop is going to be.

Too bad that the guy isn't allowed to give his talk. Then the audience could ask their questions afterwards, and as ScubaSki suggests, using a mice. This is is just a waste of time, sorry to say because it's a really interesting and important topic.

superposition scares me

Okay, so I'm misunderstanding some things about this Contrast Consistent Search setup. 1. So to get the activations associated with answering a given question yes/no (e.g. 'Are cats mammals?'), are you forcing the model to respond yes vs no? With a prompt telling it to, or something else? Or os the model just interpreting this question & answer set as the full prompt, and you're looking at those activations? Seems like any of these could bias results in different ways. 2. I think I'm misunderstanding a key point in the setup here. You have your model activations for e.g. 1) "Are cats mammals? No" and 2) "Are cats mammals? Yes". Then you map the model's activations (phi(x+) and phi(x-) to a probability of being true. You said that this is unsupervised, so that mapping is learned not from true/false labels, it's learned from ensuring that p+ ≈ 1 - p- and penalizing those two values' similarity. So, what is stopping the theta model from simply learning the activation associated with the 'Yes' and 'No' tokens? In the slides, all the x+ question/answer sets are 'yes's, and the x- ones are 'no's. That seems far, far simpler for theta to solve than figuring out if a statement is truthful. So, what am I missing here? Why is that not happening? Because clearly from the graph in 43:30, theta is learning something related to true/false, and not to just yes/no.

I look forward to serving my robot overlords and ask that they repay my loyalty by giving me my own robot servant