Видео

love the idea of noisy representation

Just recently found your channel and am enjoying the content. This video especially resonates. Since learning about the concept of embeddings, I have wondered what a visual space using the embeddings of our greatest writers would look like. Shakespeare's writings, for example, had beauty in both meaning and structure, it would be interesting to see if there is structure or beauty in an embedded space. Thanks for the great content.

Someone actually explaining the research paper code ❤

my brother from another mother. I agree completely with literally everything. And yes, fuck OpenAI.

Can LCM be combined with the Byte Latent Transformer and the Continuous Train of Thought?

uv > pip : pip install uv ; uv pip install [modules]

Thank you i have reviewed and subbed great content !!!!

I wouldn't per se it is quantum probabilities. There is some wild stuff, going on. But let's not confuse people. Good video otherwise, though. the concept space utilizing sentence atoms , is a awesome. Note, they may not be full sentences think more of an atomized sentence, without the further reduction into tokenezation. this is really cook actually, and surprised it took so long to even conisder it. the diffuser usage is kind of interesting also. Currently I'm working on a cross attention implementation to map visual atoms, (using florence2 segmentaiton) of conceptual vision. Now what's cool is the diffuser usage allows for some hackery that can be forward or reverse through diffusion for both generative and visual encoding. I've had some success with a few attempts, but it's kind of beefy...too beefy, so I am going to have to shave it off a bit and try to get it to use a lot less vram.

I really enjoyed this video and your explanations within. Subscribed and will be checking out your other content Keep it up !!

Wait what

The idea of encoding data with line intersections is interesting, however after looking at the code of the linked colab I don't think the code does what you think it does. To keep things short I'll only refer to the last iteration, the EuclideanBraidEncoder. The biggest problem is that the message is never recovered from the crossings so there is no demonstration, that the encoding actually works. Whenever the message is printed it refers to the original message variable. The next issue is, that the encode_message function just takes each crossing and assigns it a bit of the message as a label. The crossing is not altered in any way, so all the information is in the label. It is not possible to get the message back from the crossings. It is possible to get the message from the labels, but there is no need for any lines or crossings at all. A bit array of the original message would do the same. The next issue is with the find_all_crossings function. It does not find the actual intersections of lines, but just points along the lines that are closer than a threshold. when using a different resolution and/or threshold this will give completely different crossings. It may also find more than one crossing per pair of lines which should be impossible in euclidean geometry. This is in fact what happens. Each line can intersect every other line at most once. For 7 lines that is (7*(7-1))/2 = 21 crossings at most so 88 is way too many. It is however much worse, because every one of the lines goes from (-x, -y) to (x, y) for different values for x and y. It is easy to see, that the midpoint between those two points is (0,0) for all lines. That means that all seven lines actually only intersect in a single point: the origin. It would be interesting to see what a working version of this idea might actually look like

Following closely

What’s the accuracy metric measuring in this case? Whether the model can successfully predict the next (target) embedding (implying the output of interest is an entire embedding vector)? And are these numbers with respect to the training or test set?

Thanks, it actually helped me with my engeneering progect.

I dont understand anything, but it seem wonderful work

Very Interesting! I wonder what would happen if we just deactivate 50% of the neurons during inference. And see how much of model's performance degrades.

maybe the linear layers need to be trained to "aim" or "shape" the inputs to the fractal intuition space, and similarly to collect the outputs?

what the fk are u yapping about

Would this architecture still support GPU splitting? In other words would a 70b model fit onto two cards with 24 gb each If total size of model was 30-40 gb after quantitization … and could you even quantize it?

Nice vid

dude saying it passionately but i failed to understand... i wish i have bigger brain

to get a similar graph effect: what if every weight was embedded in a N-d (ex: 2 or 3) graph layout space (entirely separate from the weight's value). then use a graph layout algorithm like force directed or UMAP and compute the edges and clusters according to N-nearest neighbors. ?

That light mode made me blind at middle of the night : (

I don't get how to make it conversational. I send it "hello" and it continues with some gibberish about being a new user to some tool. It seems more like an autocomplete than some assistant. Do I have the wrong expectancies here? Ultimately I was interested in learning to talk to an AI who could call my functions.

To be honest this sounds like what I was trying to approach in terms of encrypting interactions with LLM's like ChatGPT-- I was trying to figure out how to steal the processing of their "logic clouds" without collapsing it into definite outputs-- Basically means I turn everything I am talking about into a shape, I tell chatgpt what to do with that shape, it processes the change in the shape, and that shape is returned, and then the key points of the shape is turned into symbols which then collapses into logic and reason at the level of words-- The whole idea here was to keep the central processing completely unaware of what I was doing, while giving it the fundamentals to process-- However, I was thinking about how to do this purely from a client side; where as this is doing the same thing from an architectural side-- Do I sound entirely off base?

aaaa so excited to start doing this myself. you inspire everyday!

Good job you proved something that was proved in 1919 when a solar eclipse was observed and light was bent by the suns gravity on its way to earth. Einstein predicted it was the case, anything consisting of matter distorts the space and time around it. But anti matter not being fully understood that could be a different story but it is also believed it has the same effect on space time if not an even greater effect, resulting in the un even pattern you see in our universe. Its believed to be more antimatter than matter.

Good Work

Man, love your content. Would be very exited to work with you/for you. I have intermediate level skills with building LLM agents, full stack development, and deep learning models development. Don't have to pay me, I fact I will pay you to work under you.

Interesting. There is only one reason to postulate dark energy, to explain the exponential expansion rate of the universe. The expansion was discovered in 1929. In 1998 it was discovered that the expansion is accelerating, this is when the existence of galaxies was confirmed. Electricity is drawn towards potential and the universe as a whole behaves the same way. Electricity comes into our homes because the neutral circuit provides the potential. Electricity is drawn towards grounding rods for the same reason. If something accelerates at a constant rate it will get faster and faster. If a spaceship travels at a constant 1g acceleration rate it would achieve about 95% light speed in 1 year. Physicists in the last century did not postulate dark energy because they understood that the expansion is a fundamental property of the universe. The fact that the expansion is accelerating does not invalidate their reasoning, it's what the known laws of physics would predict provided gravitational forces are not strong enough to counteract the process. All studies to find dark energy have been fruitless because it doesn't exist. To say there is dark energy is to say there is 5 fundamental forces, there is 4. All the great physicists of the last century did not postulate dark energy for these reasons.

ok i have never read into this much so i cant really say anything about this, im commenting anyway so algorithm sends you some more people who might be able to talk and think about this more

You know a lot about many different fields. What is your educational background? I would guess Comp Sci or theoretical physics?

Subscribed. Exactly the type of material I love.

It's rather fitting to say LCMs are an upgrade to LLMs. Meta has been killing it lately with its Byte Latent Transformer and Continuous Chain of Thought paper. Btw, great explanation. Learning a lot from you! Much love from India :)

What would be the accuracy with best performing/architecture (Falcon3 or Lllama3)?

you are so cool! i wanna learn more about everything. can i be an apprentice?

daddy's about to come in from creative mode to get in on this breakthrough 🤩

I don’t know what you are talking about but I am going to subscribe anyway.

you really have to explain things in detail. i didn't understand most of what you said.

Great Video, really one point. Thanks for sharing your code, very interesting

Hey Thanks a lot! I have a question, Let say i want to prepare embeddings specifically for Financial statements(10K), I can simply provide 10 bunch of FS and then index the data? will that work?

Great session, keep up the good work ❤

Thank you. This is the first time I've seen this channel, so I just subscribed. Thanks for the detailed explanation!

Keep on the good work

I just finished having a discussion with Grok about the differences between Latent Consistency Models (LCMs) and Large Concept Models (LCMs) as well as what would be involved in combining the two architectures.

Love it thanks

Wow

Did you try 1bitLLM/bitnet_b1_58-3B huggingface or 1bitLLM/bitnet_b1_58-large?

As I'm sure you realize (especially since you did bring at least one of the laws of thermodynamics), building a physical version would require discarding some well-established laws of physics, and replacing them with new ones - which is a tough sell. If you really believe that you may have somehow uncovered a perpetual energy machine, and you think that your theoretical framework could be experimentally verified in the real world, the absolute first thing you should be doing is assuming that you probably made an error somewhere, and then either publish to something like Arxiv, and then be asking those in the field to falsify your claims. Unless I'm incorrect about what I think you're implying - that whatever you seem to have uncovered in the digital/mathematical world could be built/manifested in the physical world - then the scientific method would dictate that you provide extraordinary (and in the case of unlimited energy - overwhelming) evidence before you even seriously consider that what you have here is something. The epistemologically diligent thing to do here is to assume that you are mistaken. And if the laws of thermodynamics don't deter you from exploring this path further on their own, then you would need to be rigorous about trying to figure out why/where you're wrong. Only if/when you and others have exhausted reasonable efforts to falsify your claim would it make sense to even consider that you may have something here. I think you're doing some excellent work, and I very much support and commend you on your motivations and aspirations, and your ethical reasons for doing your work. I also want to be transparent and say that I am not, and can not falsify what I'm assuming are your implications here - I do not have the advanced enough mathematics or physics education to do that. I do, however, have a solid understanding of the scientific method and its applications. I'm communicating my $0.02 to you because I think I would want that feedback if things were reversed, and even though I don't know you personally or beyond the few videos that I've watched here, I care, and I respect what you're doing in general, so I hope this feedback doesn't come across as shootdown in any way. Keep doing the amazing work and sharing your ideas my dude - and thanks for all you do!

Thanks for this we should really try building a concept embedding using high dimensional vectors