Can AI edit Twitch VODs?
HTML-код
- Опубликовано: 14 окт 2024
- TWITCH TWITCH TWITCH! If you're a streamer, this tool might help you out with highlight reels. It does a lot of the editing automatically, which is a big relief!
Check it out here: jumpcutter.com/
Also, good news! Since I animated this video, there have already been two major updates to the project that makes the workflow even more swimmingly. Specifically: We've added a much faster renderer and some tools to help make editing the highlighted sections easier.
Subscribe to AK Soccer Legend! / channel
My Abacaba Coding Tutorial in Processing:
• Abacaba-style coding t...
Lazykh explanation, (why I was gone for 6 months):
• Lazykh explanation, an...
JumpCutter.com
Discord: / discord
Developer Discord: / discord
Support: support@jumpcutter.com
Twitter: / jump_cutter
Instagram: / jump.cutter
Credits:
Go follow everybody! Links in the description
Peter - / @peterrue
Andrei Parrent - aparrent.com/
Mehmet Efe Akça - github.com/MMDF
Devin Prothero - github.com/RcC...
Caden - / torcado
Max - github.com/MxM...
Jacob - github.com/wyskoj
Bottersnike - github.com/Bot...
Garrett - github.com/Gar...
Lukas Kaufman - github.com/Lamaun
Kira - / jazzlover5972
GCK accounting - rb.gy/4jh5sk
Diogo - / diogostuart
GitHub open-source code for my prototype of this project in 2019:
github.com/car...
Subtitles provided by Erasmus Magnus! (Just one day after the video came out, omg that's fast!)
![VENEZUELA vs. ARGENTINA [1-1] | RESUMEN | ELIMINATORIAS SUDAMERICANAS | FECHA 9](http://i.ytimg.com/vi/SJfoPdeOPCQ/mqdefault.jpg)
Based on how this seems to work, it would be very easy to add a chat command which could be used to add a marker in the VODinator, so, if you're in the heat of the moment and want to mark a moment, the streamer or a moderator could enter the command into chat to make a marker.
Great Idea!
In case cary doesn't manage to see this comment, you should probably proclaim it in the Discord. so that the developers can see it.
Or even a retroactive clip. Put a command like !clipthis x and the last x minutes get added
Funnily enough twitch already has markers, that feature just seems to not be used by anyone
Cary could also fit some of the code from his other project where it automaticallly cuts RUclips videos based on noise (the companion to the VOD editor) except with markers instead of sound.
Man, Cary’s stuff like automatic lip-syncing and all that can be revolutionary for him and lots others.
I know he’s generally well-known and popular, but he still deserves so much more for all the stuff him and Michael do!
Well you’re not gonna see this but thanks so much for the heart Cary, that was the last thing I was expecting today c-:
I hope he releases the server side to do it yourself
Looossseeerrr!!! (Please don't take this out of context)
@@ScutoidStudios yeah ik lol
@@ScutoidStudios Winner.
You know it's a great app when the creator simply explains what it does and you go "WHY DIDN'T I MAKE THAT!?"
@carykh For the AI engineer, are you planning on doing NLP with the text or doing some feature engineering / more complex models for highlight detection? Also, is it possible to find where users have made clips of your content?
e
And so, Carykh goes back to his home, to fix a problem for much more productive people then me yet again.
Yep.
yea
Than*
me, yet again.*
e
I can't believe it, he exists! (on his main channel)
@EliGabRet henlo
He remembered his password
Carykh has forgotten to upload on his channel for several months, this user proceeds to exploit that fact for a comment.
The "(on his main channel)" part of his comment references the fact that carykh still uploads on his other channels.
@@fatfroggeorge2373 I would woooosh you, but you probably get the joke and are trying to ruin it.
@@DaRedPanda look at my name man
In a couple years Cary KH will have fully automated content
Well this is cool as shit.
Will be trying this out!
Thanks mate; tell us what you think, we're always looking for feedback.
Oh my god, it's SIMON CLARK!!
I imagine various Yogs would also be pretty interested in this - I hope it works as good as it looks!
Show sips!
Pog
The first video I watched from this channel was the Jumpcutter editor. I'm very glad to see this is still standing.
Yes bro!! I know what a long time coming this video (and the app) has been, and how much work has gone into them the past 2 years.
BIG BIG UPS on getting this out!
The “it’s cary tech support time” just sounds relaxing
"Editing takes so long. Let me disappear for 6 months so I can produce videos faster"
Ah, the classic automation paradox
Did not know the jumpcutter had a gui now. This is awesome!
I want to know how to make some kind of editing software like this but i don't know what tools to use. It looks like some kind of WPF gui but not sure
@@Kat21 You could start by creating a basic front-end for ffmpeg.
@@supahx1421 I've been using FFmpeg to convert video/audio files for a while, that sounds good
The day jumpcutter becomes free is when the truly anyone can become a RUclipsr
That's a big if.
Their work should be paid for developing a cool program.
I'd still have liked it to be open source tho, because it's such an interesting idea. And surely there'd still be financial support for the program
and Open Source
Carykh isn't the only one making software you know, google what you want (maybe + "github") i know silence skipper at least exists multiple times, and i saw a video about twitch compilation software that uses clips the audience takes
I like that Animated Cary and that circle guy with no limbs are just chilling in space on the JumpCutter logo
Wow, cary's gotten so much better at animating, i can't believe he animated a face this time
no hes always just been good at animating, check out jacknjellify, his first channel. It just this time he wanted to make it extra high quality and other times he couldn;t be bothered (rightfully so).
@@qualia765 its a joke
Nice first upload in 5 months, good job cary
XQC’s editor seems like he’s been using this for months
xqcL
he gets paid based on how much he uploads, so shit id be editing 24/7 too
@@Vexinal 100k views with 2/3 videos a day from 12 hours to sometimes lately every other day 24 hour streams. The amount of money the editor is making is insane with enough content to last the next year.
didnt he complain about people trying to edit his vods as fast as posible to get veiws before his team can
the sad part about trying to be competitive as an editor
Sold. Youve saved a fan so much work coming soon. Ive already been too overworked to do editting, and it soon will be worse. This is a life saver for time. This was practically a 1st year of streaming gift for me.
For non-SMB Speedrunners, a flagpole glitch is when you skip the flag coming down entirely by clipping into the block the flagpole is on. You might think hitting the bottom of the flagpole would also skip the flag coming down, but it’s the same time as if you hit it at the top of the flagpole. So when a speedrunner uses the flagpole glitch to skip the flag, they have to memorize and utilize a very very strict route for them to get the perfect clip into the flagpole block to save those frames.
Finally cary finally uploads after 50 years!
Yes
I love how he has us to be a blue head
0:20 This seems like a hint to a future video
one major thing I'd add is something that, instead of sensing chat engagement, centers around points that may be louder or have more sound
I didn't even know jumpcutter was a thing. This might sound kinda cheesy but a software like this is literally EXACTLY what I needed.
Actual lifesaver, Cary! I used to stream a lot more than I do now, and one of the big reasons is because making highlights of my streams was really taxing on my GPU and my memory. Most video files do "okay" on my PC as long as it's not super-heavy editing, but the sheer length of a livestream VOD makes Premiere a stuttery mess and it takes FOREVER to edit the videos down. Thanks for creating this
This is what I do to keep track of the highlights: Start a stopwatch when you start streaming/recording. Press the LAP button when something exciting has happened.
When I am done, I can easily see that at 0:23:00 and 1:04:12 (etc) is something I want to keep.
Smart! That could be a really cool feature in this as well- where the streamer uses a keystroke to mark moments, where it saves the last 30 seconds or so, which can be fiddled about after stream
daaamn those animations are on point. I used to do these in Adobe Flash, I know how hard to animate those.
w
4:16 Yeah, he wanted to do a flagpole glitch on 8-1 (and 1-1, 4-1, and 8-3. He does a bullet bill glitch on the 8-2 flagpole.)
Hi carykh! You're one of my role models and I hope to be able to make awesome projects like you someday, and maybe even join you at Stanford! This looks so awesome
There are so many potential cool features and measures/modes/options for this program.
Good luck!
Nice! I usually do enjoy the creative part of editing my videos, but I've been struggling to find time, so this could be an absolute Godsend if I use it in kind of a hybrid automatic/manual workflow. I'll give it a try!
Is he back?
Yes.
Is that a good sign?
*yes.*
he did not leave
he uploaded on lazykh
Not a good sign, this garbage software is paid!!! Where is open source ffs!
The Cary of KH is back :D
This is great work. Automating something like this will help a lot in the long run, you’re doing gods work.
Fun fact:
In The original portal there’s actually a cut weapon from half-life 2 named the Hopwire
cool
cool
i bought a new computer today and while installing some programs to it I suddanly remembered this channel existed and I havent watched anything from it for a long time. I searched it only to see there was no new video since when I last leave here other than this one video which uploaded only 4 days ago. this is what I call a *telepatic notification system* . I'm so glad to see youtube finaly managed to make the bell work.
The way blue Cary's eye-tracking software has the right eye correspond to the left tracker and vice versa
I can see this tool being used very often by streamers and RUclipsrs alike.
I hope this will work out just how you want Cary!
You could use soundwave (mic peaks) to correlate with chat. Exciting moments tend to be loud.
I'm convinced that Cary is just letting an A.I. run his youtube channel at this point.
Shhhhh. Carry is AI.
@@epictjkaleo482 is that a rb battles reference
Cary Tech Support should’ve been called Cary Komputer Help
Looks like the legend himself is back, after a whole bunch of months.
When the world needed him most, he came back.
being a regular user of jumpcutter (I use it to cut down recorded zoom lectures) I am glad to see that it got an update :) thank you cary!
Im not sure how easy it would be to implement, but Twitchtracker also graphs the number of clips created, which for large enough streams would also be an additional indicator. And showing category changes for variety streamers might be useful
Some ideas for this:
add a way for the streamer to mark a part that they really want to be in the video (for example: their intro explaining the idea of the video)
add a way to automatically add intro/outro videos at the start and end respectively
add a way to specifically filter on certain types of reactions (for example, on twitch a peak in OMEGALULs is a very different clip from a peak in Fs or RIPs)
make different 'peak' activation strategies, one of which could exponentially die off, such that there is never a cut that is 'too big' (in other words, the time in the VOD between clips is never too much)
you could also look into temporal machine learning algorithms (I worked with TCNs before for music structure annotation), I think they would work extremely well for this, with spectrograms where the bins represent twitch emotes or common phrases
good luck with this!
YOU'RE BACK!!! THE LEGEND,THE CREATOR, THE ANIMATOR!!!
How the hell did I not come across this sooner, despite the fact I've been looking for something like this for ages
Very consistent uploads!
I’m just here because I love your videos and I’m glad you posted again after 5 months
Wow, had no idea this was professionally going on; very cool stuff!
I'm with you!
I will try to do my best to help in the project as a developer!
I've chatted to Bottersnike before (Helped me configure my DNS, nice guy), but I never knew that this was a project he was working on - the shout-out blew my mind! I was thinking in my head, "Hey, I know this guy!"
0:21 his eye-beams cross, but his eyes don't.
I was in the test stream :O
Feature suggestion: If it's not already in the works, an auto-captioner (like the ones you do in this video) could be easy when integrated with Google APIs for speech recognition. It'd also be incredibly useful!
the way you showed here (0:43) would make a great UI feature to set the threshold for the AI to select clips from
You should consider making it free and open source so that the community can also add features
Many people worked on it, it's fair that they want to get some reward for all this work. If you want them to add something ask on discord
There are already people contributing full on tools on our discord! Join us at the link in the description. It’s a very chill community!
I will literally use this to watch Vods of all my favorite streamers in the time it would take me to watch one normally :D
Really hope this Jumpcutter thing gets a mobile version one day, it'd be EXTREMLEY useful for me
At this point cary can be a teacher
Ayyy you’re back!
Cool project. The only thing that bothers me is the annual pricing. I would even pay 120$ for it once but I can't afford to pay 60$ every year.
I would even dispense with new additional features als long as I would own the main software for 120$
Unfortunately, I have implemented "exactly" what carykh implemented about 2-3 years ago but It really didn't work out well in production (I have requested some editors to work with it as a "highlight finder").
It worked well if you have decent amount of people watching the stream doing JustChatting stuff, but it didn't work well on more "energetic" or "interactive" streamers, especially if people are "spamming" the chats on streamer requesting interactions. (I have implemented dynamically shifting chat threshold, but It didn't work quite well, missing several highlights that human could find) by the way, very nice for carykh for doing job for his own!
is yours on github? looking at the one from this video, it just removes silence, and the official version is abandoned. I think the idea of a fully automatic editor might not be feasible yet, but even having highlights in an XML or the full video with nothing in between highlights would be helpful to speed up the editing process
I think this one is gonna have a big impact...
“who wants to be a leech?” *proceeds to use someone else’s vods anyways*
I am sending this over to my brother-in-law, a twitch streamer. several people have done Highlights of his streams, But this would be much easier.
Edit what the Ai finds, add in any memorable bits and DONE! I likey.
I already made a prototype of something similar although with different functionality. It didn't have the editing part, but it showed graphs based on engagement, but also the type (which was based on emotes used). But the biggest issue was the channel-to-channel difference in understanding and usage of emotes.
For the jump cutter for speeding up educational videos, consider increasing frame rate of speed up material. (like 60fps minimum)
Someone else made browser extension with jump cutter functionality! It is called "skip silence" and works in youtube and other html5 video sources. And with silence player it works on local videos too.
This seems useful, can't wait to see what you do next Cary!
This is really smart, well done!
Finally you uploaded after 5 months!
Okay, the project looks great but 5 dollars a month..?
Can't we just buy the full product for, I don't know, like 50 bucks or something?
ehh, it makes sense in this case since the software delegates work to their servers; you're subscribing to server access
@@JumboDS64 Maybe they should move that code to the client then, because currently the tool will stop working if the servers ever close
I’m also interested in this tool, but I dont think the subscription model would work for me(I’d rather use this tool to quickly find the clips so I can edit them quickly rather than having a video encoded immediately). I’d be fine with having to pay for feature updates if it means having a one time purchase license to own a copy.
Subscription model on a beta project. Stunning and brave.
YES NEW VIDEO
Good stuff, keep working on jumpcutter I love it.
wonderful project, can't wait to see it progress ❤
You should add an auto animated webcam that automatically generates captions and then either with a preset or by drawing a character and their faces makes an animated guy that talks for you
0:20 i like how their eyes are connected to the opposite part of the reader also the face cary makes
0:37 PHINIAS AND FERB REFRENCE
This is just an intermediate step for Carykh's ultimate plan to create an AI streamer that is more entertaining than any human.
Shoutouts to Miniland333 who got featured in 0:04 of the video POGGGGGGGGGGGG, and Illumina as well POGGGGGGGGGGGGGGGGGG
In all seriousness tho, it's very very very cool to see big RUclipsrs supporting speedrunning, especially SMB1 speedrunning AND Minecraft which I both love :) speedrunning is poggers. Wen cary keyboarding hastily?
Carykh is on his way to becoming a legend. So cool!
I JUST STARTED THIS, THANK YOU SO MUCH CARY
Yay! A new Carykh video after 5 months!
I wish i was a billionaire so I could throw momey at this dude. I must work harder! 😂
Haha I actually thought about this when you first announced JumpCutter, but thought it would be too hard to implement myself.
finally some AI supported tools for everyday tasks
This is pretty damn cool. Unfortunately (but understandably) it's reliant on the chat, otherwise it would straight up automatically edit straight footage from nothing, which would be insane. I'm thinking you might get a similar result by analysing the audio and when ai gets better at audio, it could tell when you're most excited, and then edit around that, figuring out where segments lie e.g. a slow build up to a peak, with various 'important' lines of speech, or just the sole moment where you are excited.
The, to summarize, there is a new iteration of Jumpcutter that is now proprietary? Kind of disappointed about the outcome, but here's for the best, I guess.
I like how we can hear the "click" every few seconds.
The early bird gets the early pin and heart! And the late one uhh.... F for respect for the late one. Anyway, nice video! Very interesting!
Also, you were right! AK Soccer Legend needs to be known!
This will help my channel once the RUclips version comes out!!
THAT'S a great idea Cary
I was waiting for a trial version, I really wanted to try it this whole time (obviously I still use command lines but I would really like to see how the flow is with a proper gui)
Cary Keeps Homingmissilesathome
Maybe you could detect other things than just chat activity.
Audio volume, bits, subs, anything really. And then allow the user to select weights for it, e.g. a chess stream won't highlight the same as a cs:go match and might need to be weighted towards one statistic more than another.
Man Cary I see you on the minecraft speedrunning binge
CARYKH UPLOAD POG