LOSSLESS Spectral Rotation : A better method!
HTML-код
- Опубликовано: 26 сен 2024
- Finally, a completely clean way to achieve a 90 degree rotation with no loss!
this method is a perfectly clean way to do a 90 degree spectral rotation, spectral inversion and time-frequency swap without losing information
Relevant video : • 90 degree rotations : ...
now my thought process was "haha what if i just took the FFT of an entire audio signal and saved that out" to realise that the FFT is a complex function, and that if i wanted real output i'd need to sacrifice information (lose the imaginary part of every coefficient) and thusly lose most phase information. fortunately i did a tad bit of screwing around and i have found a way to do it, i went down a few too many rabbit holes and got lost in some complicated notation but with enough rock-bashing i managed to concoct a little script to flawlessly spectrally rotate an audio file by 90 degrees
also looked into this "hermitian symmetry" stuff but only to realise i was dealing with a real-valued signal all along (duh) aha~
you can find the script here : gist.github.co...
for requirements and stuff do pip install scipy numpy soundfile
and if you're getting that one cffi backend error may all deities help you because i've never seen that in my life
skibidi
skibidi
skbidi
skibidi
skibidi
skbidi
it turns drum breaks into chords…. intervals in time become intervals in tone! this is so cool
I have never seen anyone present something more chaotically! The level of zooming and scrolling alone, without the voice, could drive even the sanest person to insanity :)
It's not a complaint though.
haha yeah this dude is fast
my adhd is a force to be reckoned with
Oh hey it's Chris! love your work in the RoR games
@@idiotinium perfect speed for me
i love these funny audio editing techniques
"you will never hear this error under any circumstances"
"this is what the error sounds like"
after normalizing
If you boost the volume of "silence" x1000 it would sound like earrape
Finally, now I can turn entire movies into drum samples
THE MAD LAD DID IT!!
thanks for doing these fun little tutorials, it's so refreshing to have a tutorial that's just so explorative instead of "how to make this sound you've heard before"
in fairness, it makes sense to make tutorials about recreation because it teaches you general technique
i shouldve paid attention in math
fourier transform is quite simple i think you could learn it!
@idiotinium in fairness is the Fourier Transform basic math? I only know it because it was mentioned in CS once
@@illford well i think its simple. what's CS?
@@idiotinium computer science
After seeing the last video I took my own shot at it with wavelet transforms (via PyWavelets) but ran into some issues and didn't complete it (shoulda gone FFT). Great to see this working!
The greatest ideas are the simplest ones 😉
Yes, concatenating a signal with a reversed copy of itself will make its Fourier transform real-only -- then you can take only one half of the spectrum.
I think all audio editing software should have this effect available just one click away, since it makes other effects more powerful (e.g. you could use volume envelope and FFT filter interchangeably).
YEEEEAHHHHHH absolutely
That's so simple, yet totally crazy. I love it!
THAT ERROR BOOSTED SOUNDS SO COOL THOUGH!
this entire channel is peak ngl
NO. YOU'RE NOT OVER. I NEED MORE OF THEEEESE.
Thanks for sharing
Nice that you included this part 6:27 , usually people leave errors and mistakes out, but I find it just as interesting to see what aspects one might get wrong when trying to figure out stuff like this. I feel like it adds more educational value. kudos 👍
just discovered your channel, can already say it's a youtube favorite :) very curious to see what you'll bring next!
the error sounds cool as hell
i knew rel was smart but this stuff is _insane_
also crazy seeing _several_ god-tier artists under these videos. she's going places~!
teehee
Yo thank you so much for keeping us updated on this. Tried doing this in ableton the other day and I objectively sucked at it, haahha! glad you're showing us these different ways dude, means a lot, very inspirational
also love how mathsy it was at the end there
I think Dan Worrall would enjoy this, and I hope you take this as the compliment it is.
this is my favorite series on yt
thank you for giving us the script!
What you're doing is essentially a fourier transformation, which is what convolution plugins do internally. What a convolution plugin is doing is taking the fourier transformation of two sources and multiplying them together before assembling it back to regular audio. Pretty cool stuff.
yes, i am doing a fourier transform
no, convolution plugins do it differently, since how are you convolving the entire signal live with an fft of the entire signal when the end part hasn't even played yet. it works a little different than that
its like theres a sliding buffer of audio thats continually being multplied by the impulse response and summed to give the current amplitude
The first error sounds like the universe trying to speak.
peak content
why does the error/loss gained up sound like insane etherial choir
it blurs the entire thing so its like infinite reverb
This rabbithole is getting deeper
Just... Just wow, always dreamed about something like that.
Using this to make my own producer tag -- I just threw my producer name in an image to audio function online and then spectrally rotate it to shorten it and post-process to make it sound alright as a stamp of your name on your track
That "noise" at the end of the file at around would be interesting to explore as an IR for convolution reverb.
When I heard "let's name this skibidi" said so casuallyy and unexpected I laughed my face off. Bro yo r a killa cooker
Id wondered if this was possible for years and now I'm very happy
you are a LEGEND.
TOKI PONA MENTIONED!!!! LETS GOOOOO!!!!
TOKI PONA LI LON AAAAAAAAAAAAAAAAAAAA
skibidi 😭😭
Biggest news of the week!!
I think RUclips unsubscribed me because I've been WAITING for the next rabbithole so bad
This is actually crazy
Nice one!
Yes! The saga continues!
"The greatest living musician" is always from the US as if you know all great musicians from all other countries. Any trained pianist outside of jazz bubble know that playing an ostinato like that isn't really THAT hard
this needs to be a vst plugin
You absolutely rule!
sorry kaixo and unfamilliar, i beat you to it~
Except I am making a VST plugin with a UI that lets you drag drop samples.
@@KaixoYES! I was about to comment that someone should wrap this into a plugin and I'm glad to hear that somebody is already on it. GO KAIXO!
woaw
I would be firs to buy a plug-in for Mac. Just make sure it can do like the samples and romplers from 90s - place inverted signal above Nyquist to the infinity, so far I’m doing with a bit crusher and a filter.
@@Kaixo what are you using to create a plugin? I am trying to learn how to make a plugin because i have an idea for a somewhat unique synth/sampler thing. I found something called JUCE that claims to be for plugin making.
Dan Worrall crying tears of joy from his grave rn
The upside-down version begs to be frequency shifted down, or just pitched/slowed down.
I tried doing this last night. Since frequency, pitch, and perceived volume all have different relationships, most of the energy and information in a piece of audio kinda just gets bunched up near nyquist when you flip it upside down, and so when you pitch/frequency shift you just move what is mostly a bunch of smushy 22KHz ringing down to some smushy ringing at some other arbitrary frequency. frequency shifting seems more useful since it spreads the frequency relationships down so the peak is less pronounced but it's still there.
Found that some extreme EQing to flatten the response gives the best results, using it to make some interesting glitchy percussion atm.
@@Curly_Music Sweet! Another thought that crosses my mind theoretically is then using that frequency shifted upside-down percussion as a vocoder modulator signal (potentially with some compression if necessary). Perhaps with the original percussion as carrier if you wanna get real masturbatory with it.
I have feelings for you
i'm flattered, although my wife says she wants the relationship to remain monogamous
@@idiotinium haha yeah I was just kidding haha. I want you.
floating point error type beat
I mentioned this in the original video lol. Note that the STFT is only completely invertible (recovering the original signal) if and only if the window and hopsize is chosen correctly. Specifically, the windows must adhere to COLA constraints (constant overlap and add). Additionally, a STFT is always invertible if the hopsize is 1 sample. Check MATLABs documentation on phase vocoding for more info on this.
P.S., the Fourier transform of a signal is real if an only if it has even symmetry: f(t)=f(-t), which has some interesting implications with an FFT due to periodic extension - the one half of the audio buffer must be a mirrored copy of the remaining half.
does that work on fractional fourier transforms?, im trying to get a real-only output from a fractional fourier transform lol
wow directory opus. good man, good man.
Thanks for code! That's useful for some things I like to do. ;)
Could you parametrise the script to allow an arbitrary rotation (degrees and axis of rotation)? And to choose between linear and log scale for each axis (thereby potentially translating rhythmic patterns into notes rather than semi-arbitrary frequencies).
Even if the rotation is lossy due to clipping, it could be musically useful for rotating effects (rotate-effect-unrotate).
i am desperately trying to make some arbitrary rotation thing for it but my brain is too tiny to comprehend the order of chirps to multiply and whichwaywhatever
expect it next year
Haha I can only imagine. Good luck!
Also im prepared to sound dumb here...
BUT NOW THAT U CONQUERED THE X AND Y ROTATION....WHAT ABOUT THE ZZZZZZZZZZZZZZZZZZZZ????? WE DOING AMPLITUDE ROTATION BABEEEE??
maybe one day!
perhaps not even by me~
Putting all this together with some kind of fancy coding and we got a 3d spectral view of the sound waves right?
Idk if yoi've seen this already but i just found a video from 15 years ago on a program called photosounder that treats sound as images and also let's you rotate spectrograms
i've never seen anyone being good at using audacity before lol. can you make a video about the math used and how it's different from the convolver frequency shifter combo?
For the test audio you should do a single piano note. That way we can get a better idea of what becomes what when it rotates.
you'll get a bunch of clicking sounds for every harmonic in the sample!
you should try it its really fun
Is there a way to squeeze the audio into a specific frequency range?
i think if you pad it with a ton of silence on both sides before rotating you can, except you would then have a longer signal
@@idiotinium I’ll have to try it, thank you.
Is it posible to write a reaper script and process the audio file in place? That would be much more convenient.
wonder if you could try making the highest end lower for the vertically flipped one so its a bit lower than just super high pitch
like squish it? maybe probably
So if u take a spectrum, flip it 90 degrees, add some silence at the end/cut off a bit of the end. You can create bang on brickwall filters?
yeah pretty much
just some strategic mutes throughout the sample and you can just remove anything you want
You should make a video going through what you did. It’d be cool to see you do some music dsp with Python/otherwise.
amazing!
I'm guessing that you applied the following iteratively (1x, 2x, 3x, 4x) to get your results: reverse the signal and calculate the DFT. If that's the case, how did you store the result of the first DFT, given that it will generate complex numbers to represent amplitude and phase, and given that the results must contain all information to eventually get the original signal back? Also, if it's the case, you could also generate a less lossy 3rd output by just doing inverse DFT on the original and reversing it.
im actually feeding the entire signal into the inverse fourier transform since that gives me real output straight up
additionally no its not iterative i just have a thing to flip the signal in the time domain which then flips it in the frequency domain
and then back unflipped for the inverted one
something worth trying for vjing
ngl normalized error (phase diff) sounds a bit scary
Hell yeah!!!!
can you specify the rotation angle? be curious what 45 deg or so would sound like, or is that just pitch shifting
Incredible work. Will this work with Mac?
if you can run python on a mac, sure, but idk how to do that since i've never used python on a mac
Thank ya 🙏
I don't do programming at all, would you ever make an simple GUI for it?
kaixo already did that and made it a vst
github.com/KaixoCode/SpectralRotator/releases
Nice thing about Audacity is you can just use an old version when they make a bad decision lol
yea thats true except theyre gonna have fun features one day, ive seen them tease that spectral brush eraser thingo and i really want that
I need a tutorial on how to make it work, I know nothing about python pls I just want it to work pls :(
sdr sharp be like
I know it would result in lower quality, as basically, you are downsampling. But do you think that maybe doing the spectral rotation on a frequency range that's within the general range of human hearing (like 20Hz to 17kHz to be safe) might be an idea? The main reason would be so that nothing in the audio would have no auditory representation at certain steps along the rotation process.
But perhaps that wouldn't make much of a difference to the sound and not be worth it.
keep up the cool work! love watching these experiments.
well it goes from 0hz to nyquist sooooooo
you'd have to simply change your sample rate to something where the nyquist frequency is 17k
YO please the final boss of this rotation pipeline would be if the script could be integrated into Edison in FL (the 'run script' option there). This would make the process very practical n creatively efficient in terms of trying out various effects on the rotated versions
(- record audio into Edison, rotate, dragndrop into fl n try things out, record back into Edison, rotate back) (instead of dealing with file management and renaming and stuff)
Whoever reading is capable of doing this (integrating the script into Edison), this is your moment!
haha that would be cool
@@donotoliver I'm working on a VST plugin that does this, allows drag and dropping of files. Hoping to get a first working version out tomorrow.
@@Kaixo And you did, congrats!
@@Kaixo yoo where can i try it?
could this be rewritten in nyquist and be native to audacity?
idk how to write nyquist
i'd love to hear you explain the math behind this
Fourier Transform
its a function in math that takes a bunch of numbers and spits out a bunch of other numbers that tell you how to add up waves to re-create that initial list of numbers
it has some very special properties that also allow for this (whatever i'm doing in the video) to happen
although you gotta set it up right such that the input is symmetrical in some specific way and then you can do the funny swap between time and frequency
was thinking of something a little more in depth but wharever
how to run this script on mac? please help!
i've never used python on a mac, but theres probably some tutorials out there
Wait aren't you the guy from Au5some fam showing desmos stuff!???
yea
Is the error just simple "issues" due to floating point math?
yea
Why when I run the script does it just blink without doing anything? :(
(I already installed the necessary modules)
i have no clue
@@idiotinium XD
Pioneer.
Skibidi.
Um I don’t think negligible counts as lossless? I mean if it would only be perfect with infinite precision then… I don’t know lossless *feels* wrong
bro its literally a floating point error you cant get much better than that, maybe i should say perceptually lossless lmao
-155dB is practically inaudible
i love how useless this is, so pure
I believe the 1 bit of loss is the dithering.
ah yeah could be that too
@@idiotinium in fact I'm 99% sure, the silence you left at the end of one of the clips would signify that!
Sounds like snake jazz
god bless
Will there be a Video where you go over the Mathematical / Code Implementation part of this or is that left as an excercise for the viewer? Anyway I enjoy this alot since i've just recently learned about a lot of the mathematical/DSP-concepts involved in this and would have never thought about such an application. Godspeed to you
idk if i could explain it well enough cus i just barely know what im doing lol
@@idiotinium can you try tho
or maybe ask someone to help explain it? maybe a math explainer would be happy to work on this
I'd like to aswell, since I'm in math and computer science (and do music), so maaaaybe me?
i will start my evening by praying to my matt parker shrine in hopes the answer cometh
@@idiotinium lolol
but really though, I think it'd be fun to try to explain!
can you tell a bit more abt how the code works
fourier transform
So, what will happen when you encode the rotated audio with lossless codec and then rotate back the compressed audio? Can this technique make the compressed audio better sounding?
instead of losing frequency information, you then lose time information, mesning youll have a spot where its just silence
@@idiotinium okay, but apart from that silent spot, the rest will sound better because it didn't loose freq info?
@@sypialnia_studio yep
@@sypialnia_studio yep
@@sypialnia_studio yep
could you make a pluggin out of it ?
Either I'm doing something magical, or I've just figured out that you could just get the dct() of your input, scale down the output of the dct, and write those dct coefficients as audio.
conserving phase information?
@@idiotinium No, using dct instead of fft just gives you real-valued data. Plus, its value are smaller than fft's
hehe
@@idiotinium This is some MATLAB code I just wrote:
[input, fs] = audioread("input.wav");
buf = dct(flip(input));
buf /= max(abs(buf));
audiowrite("output.wav", buf, fs);
Thats it!
banger ctf chall
Is there any use for that technique?
definitely
So, Watermarks now really usable ?
I'm looking to do something similar in JUCE with realtime audio but I struggle with the math. Would you like to collaborate?
i barely have any idea of what im doing, also you can't really do that with realtime audio
@idiotinium I'm certain that I can do it in real time, albeit with a large memory footprint.
well yea """"realtime"""" cus you first gotta get the whole sample into memory and then run the thing
@@idiotinium You can take FFT's at regular intervals and stitch them together to form a continuous spectrogram.
The idea then would be to represent the cursor as two points on a 2D plane rather than a single vertical line.
Well I did it. There's a fair bit of aliasing but it works.
Goated
no way, undertale-yellow-v1.1
aha i still gotta play that
Hell yeah