I learned about Streams in 1998 in a security training class. They can be used to hide any file, even malicious files. anti virus scanners don't check for files in streams.
You said don’t check files in stream why? So these hidden files should I delete,I just bought the surface go 13 inches. The only thing I can’t find is the service key it’s a long # where do I find it? I thank you your so informative & very intelligent person. You are AWESOME
I made my living in IT for over 25 years. Started out with standard desktop break-->fix work and worked my way up to Enterprise WAN management. I've also done some scripting and programming. I would easily fit into the "expert" category (though I never liked calling myself that) and was often referred to as such - if someone was told to call the expert, they'd be calling me. In all those years, I never even heard of alternate data streams in Windows. Like they say, "you learn something new every day." My hat comes off to you for even finding this feature and then explaining it, well, "expertly." Well done!
I've worked in IT for about ten years and know stuff about Windows that even some of the true graybeards around here didn't know... I also didn't know about this. Windows is such an insanely deep rabbit hole.
I've worked in it for as long as OP and DID know about ADS's. They have some practical uses but mostly exist as optional features unused by most users and software. The fact you can use traditional CLI commands to manipulate them is a little remarkable. I thought that would have been handled mostly through Powershell. This is a fascinating presentation.
Grey beard here, knew all about ADS and played with it, hiding stuff in there, cleaning virus out of that space, etc. More utilized under Apple products, but I was doing Windows. You'll find ADS in your saved URLs, zone information is saved in there, even to this day. I usually clean up poisoned URLs so they are normal 2 digit size, not 3 digits in size. I'm sure there's other things I don't know but I've seemed to have forgotten it. 😊
@@fllthdcrb When was the last time you "on practice" accessed a data stream as a file? Yes, never, thank you. "Security threat" - that's adorable. OS is aware of them, file system is aware of them, antiviruses are aware of them, IT pros are aware of them... the mere fact that YOU aren't, does not make 'em a security threat lol. It's a file system feature which may be used by malware - just like any other OS feature - e.g. you know, malware is usually stored in (drumroll) files! This does not make files a security threat.
@@hackdesigner >When was the last time you "on practice" accessed a data stream as a file? That is literally how you access data streams. >Yes, never, thank you. You know when you access a file? Psyche, you're accessing the file's default data stream. >"Security threat" - that's adorable. About half of google search results are security blogs.
You can also use WSL to create a folder NUL or use parted magic to create folder with forbidden characters or use UNC paths to create folders ending with space alongside a folder not ending with space.
As a noobie data-hoarder, the Zone Identifier seems extremely useful. It's a shame that it's hidden away like that, I really like the idea of files keeping track of their origin.
On the contrary, I was always annoyed by this feature, so at some point I looked for a way to disable it and now I always disable it. I've known about alternative streams for a long time, but I didn't know that the zone identifiers are stored in them
Usually when I hear of "super hidden" files, it has to do with a completely separate concept from Alternative Data Streams (available as far back as Windows NT) which are the desktop ini files. These alter how Explorer shows (or hides) the current folder (and sometimes even its parent folder!) in every Windows version since Win 95. The most notable example is the recycle bin. You can see its true structure inside cmd or winfile (if available) but not within Windows Explorer.
@Evi1 M4chine On older versions of Windows (95 for instance), they'd refuse to display certain folders in explorer no matter which UI options were toggled. You'd have to launch explorer from a command window to get around it (or just use the command window itself or the old File Manager.) Somewhere along the way, they gave more sane options to display these, at the expense of hiding other stuff (including alternative data streams.)
you teach me something pretty much every time i upload. hopefully one day i can keep this knowledge heading downstream and help other people. thank you!
I work in IT and did not even know this. I also always wondered how the OS would know a file came from the internet. I thought it might be some database or registry somewhere but it was odd because when I copy the file to another computer this would stay with the file...now I know. Thank you.
When we first launched NTFS we had a whole section in our 5-day training course, dedicated to streams & how to access them. The vision of the dev team was quite vast. 1. They'd let you extend the NTFS properties, so you could add your own custom tags needed to support an advanced Document management system. each file became the equivalent of a row in a doc database. 2. You could do version control. Keeping the delta between the version as streams. 3. You could do translations. The doc could contain streams for French, Spanish, English etc. & your application opens what it likes. 4. Similar to (1) Audio & Video could be tagged with production attributes commonly used in radio or TV production. Unsurprisingly it never took off. Most Architects & developers don't get close enough to the API's to really understand how extensible & powerful the Microsoft products really are. So they don't think to use them, or they design some complex system to do the same thing the product does better out-of-the-box. I'm not saying that is their fault. There is only so many hours you can devote to really learning a product.
Another great thing are Junctions. I used them on a Win2K fileserver to build a nice tree from different partitions and shares. 20yrs later they are still not editable by UI but implemented in CLI commands.
I knew about this when I checked out the alternate streams option in 7zip, in which I found the zone.identifier file, and now you have confirmed those things do exist
Metadata can be attached more easily is my guess; most programers dont want to bother with adding additional files to a files attributes to attach them to said file.
More likely is the fact that it hides the data by design, kind of defeats the point of trying to keep track of version history if you obfuscate the data!
@@Fifury161 Well the idea would be that the application that uses ADS to store version info would also have the ability to scan ADS to retrieve it. The obfuscation in this case is actually useful because (if combined with encryption) it can prevent people from tampering with the versions.
@@sophiacristina Right?!?! oh gosh, I wasn't that intelligent. I found out how to enable dictation and then plugged in my radio to the aux input and let the poor (according to today's standards) dictation engine type away seemingly random phrases and make me a "story" lol. I did find a Java page and the hidden Media songs but I had no idea what to do with those. Microsoft put so many surprises into these wonderful machines
@Sophia Cristina XD Aw thank you! It was super fun. In a way I would totally do it again if the dictation programs today would type words out if it heard random static and music. That's a great idea, I think I'm going to give it another shot on this computer. Thank you Sophia! :D
This info was shown in a German "Computer Bild" PC magazine around 1995, when I was still in Highschool. I'd never have thought that this "feature" would be carried on until today, as it didn't make any sense back then - and it does not make any sense today (except another target of viruses).
I used to use a program that made fake "bad sectors" on the disk, and then wrote to those. Only a few advanced forensics programs can find them, and if encrypted in a certain way, even that can be ofuscated. Worked on all operating systems.
I wouldn't put too much stock in the idea that this is ignored by forensics software though. But it could be a useful way to hide personal info in a way obtuse enough that malicious actors would fail to find it. Basically, don't use it to do illegal things, but it might be useful for stashing say, a crypto wallet's key or keypairs.
@@stevem1097 I think I figured out why there's no software names - every time i put any in, the comment gets nuked. There's two things you want to do to accomplish this. First, manually and arbitrarily mark bad sectors, then read/write to sectors using low level disk access. If you google both topics you should find a few solutions pretty easy (the ones I can't seem to share with you for some reason) Be CAREFUL if you mess with this. You're not working at a level where partitions and files exist anymore. You can seriously mess up your system.
@@stevem1097 I would also suggest that you think about process first, software second, in the future - considering how the comment reads, the actual piece of software, if the original commenter even remembers the name, might very well not exist or be deprecated by now. But if you know what you're trying to accomplish and can translate it into process, you can then attack each step of the problem individually
OS/2 has a nice use for this. It uses REXX as a scripting language (like Windows uses PowerShell), but that is a compiled language. It will be compiled automatically when you start a REXX file, but that could takes several seconds back in the day. To speed this up, the compiled version i saved in a data stream. So starting a REXX program is only slow the very first time you run it.
One problem with Windows, and I suspect OS/2 as well is that filesystem features are too closely tied with one particular volume format, namely NTFS in the one case, HPFS in the other. On Linux, you have a VFS layer in the kernel which supports the use of a whole range of filesystems, including compatibility drivers for the old ones from DOS, Windows, and OS/2. The trouble with the “data stream” concept is that it does too much, and yet too little. It’s like a directory which can only contain files to one level, and with limited names: why not replace it with an _actual_ directory?
@@lawrencedoliveiro9104 That's what OSX does. (On the other hand, you cannot install OSX on FAT32, so that workaround is useless...) But the issue is something else: You cannot innovate if you want to stay compatible with the oldest technology ever made. At some point you want your operating system to have a filesystem that can do more than FAT8 with 6.3 character ASCII filenames, 8MB file size limit and no directories. And when adding more and more features into your filesystem (last access time? check. last modification time? check. creation time? whatever. owner? go on. group? ... acls?) you will come to the point where it stops making sense to add specific fields for each and every possible data value. And at that point the only logical solution is to have a generic structure, Extended Attributes in OS/2, File Streams in Windows. Both are basically simple key-value lists. The real interoperability issue here is that there is no way to store those on FAT volumes. So every operating system finds their own way to store them, and every one does it differently. In theory all operating systems could use the corresponding storage system of any filesystem like their own, if they had drivers for those. But they generally don't, so we end up with all that garbage on FAT volumes.
That's a good use of these, for non-critical information or as a cache, since these alternate data streams generally aren't preserved when sending the file across the Internet.
When the input file contains a [Ctrl]+"Z" character the TYPE command stops there with displaying text. Typically a remnant of the MS-DOS days. I suggest using the COPY /B variant to add an alternative data stream.
Well shit, this is actually something I was concerned about. I was wondering why there was so little space on my pc when I've barely downloaded anything. Maybe I accidentally downloaded a virus.
that space is most likely held hostage by windows itself. If you have hibernation enabled, there'll be a hibernation file about the size of your system memory where windows dumps its RAM contents before it goes to sleep. So on a system with 16GB of RAM that's 16+GB of your SSD gone. If you haven't configured your system memory manually then windows will create a swap file that can be anywhere from 1GB to 150% of your machine's ram size. A 5GB swap file is pretty normal with memory management set to auto but it can grow much larger than that. My workstation has 128GB of RAM and for normal usage a swap file isn't necessary at all since memory usage is only 10 or maybe 20% ... but left unchecked windows wont hesitate to create a 40-60 GB swap file ... which is of course of no actual use and only slows my system down while eating away at my SSD. By default windows also reserves a considerable portion of disk space for windows update downloads and update backups/ update roll-backs. I think after my last fresh install it was already about 8GB or so. System restore points also take up a lot of space if you create them regularly / before every software installation - which is a good thing to do, but managing restore points and deleting old ones will free up maaany GB instantly. No need to keep more than the latest two restore points really. It's also always good to check the systems temp folder(s) once in a while ... sometimes install packages aren't deleted on shut down for some reason and then linger for quite a while. WIndows by default will also use it's oh so great prefetch and superfetch functions to pre-load/ cache data of frequently used software to speed up loading times. Obviously useless with Sata and NVMe SSD system drives. Windows is a pain in flank these days. On a vanilla W10/11 install with all default settings losing 50-100GB of disk capacity to Microsoft's nonsense is normal.
This is honestly SUPER useful for hiding watermarks in any art files that are sold, such as 3D models. Makes it easy to prove ownership if someone tries reposting it as their own.
Great call on WizTree. When we used Kaspersky at work several years ago some log file it created used up all the space on my hard drive. WizTree was the first utility I ran that found what had used the space. We thankfully no longer use Kaspersky, but I bought a personal license for WizTree.
Great job on a topic relatively few windows users know about. Those who are security focused will know about this, but the tools you present to find data streams are good for general users. Well done.
Heh, extracting data from Mac files on an NT 4.0 server was exactly how I learned about streams. I hacked together a few utilities & menu items so I can quickly see where downloaded files originally came from. Lots of cool things, and lots of programs like Firefox, Notepad++ etc. can get at the streams if you know they exist. I've had way too much fun messing around with them.
I went into this video thinking I wouldn't learn much as I was already familiar with ADS, but you taught me a lot! Very thorough and informative video! Thanks a lot man!
It's an NTFS/ReFS/UDF attribute, so it wouldn't work in most cases as this information isn't converted/preserved when copying the file into a zip archive, website, non Microsoft file system, etc. If the ARG consists of a program that install stuff on the computer, then sure this could work as the program can modify/add the alternate data stream attributes but if say the ARG had an image that you'd download off the internet, you can't hide additional information in it. You can only store the contents of the file itself and the file name (not even metadata like permissions is possible). Often a lot of programs like file archivers will retrieve & convert attributes like permissions & access timestamps from the FS to store it in its archive format and then convert & apply it on other systems when a file is extracted, like Windows permissions being read and then stored in the zip archive format which when sent to and extracted on a Linux system, the file archiver will set the extracted file's permissions & access timestamps with the data stored in the archive format during extraction. Alternate data streams are not one of these attributes that most programs preserve as they're pretty specific to Windows systems, unlike more universal things like permissions and access timestamps.
A platform independent alternative is hiding a zip file insize an image file. Note that this will add to the reported file size and depending on how big the zip file is compared to the expected image size from its quality and all, some people may catch the discrepancy and look into it. Participated in an event where they did exactly that. I believe the practice is called Steganography.
@@Vixel4076 yeah, steganography basically is "hiding a message within another message or object" though depending on which angle you look at it, if you look from the file angle, it is steganography, but if you look from the image side some may not consider it steganography because if you screenshot or convert it losslessly, the hidden file disappears. A decent way to hide info is also within docx files, which are basically zip files with some structure, and can be made fairly large if they contain pictures, so a few hundred kb can pass fairly easily. For larger files, a lot of videogames use a compressed file, sometimes encrypted (but since the game needs to open it, it is possible to get the key), so you can hide a couple gigabytes of files in there.
Keep up the good work As a avid Linux user I decided to jump into windows and tackle it and learn as much as I can learn and you’re really a lifesaver for me thank you for the video’s
By the way, Thio, I'm about to use your YTSpammerPurge tool right now, because I'm starting to get really annoyed by spammy comments, so I figured I'd do some "janitor" work in my free time for some RUclipsrs I admire. Great job, bro! 👏🏻
See, this is why I always end up going back to watch a video even when the title isn't "clickbait-y" enough, and I feel so damn lucky that I have the tolerance to sit through anything, and in the end, it's very rewarding 99% of the time. I wonder if this alternate data stream thingie is related to an executable I used 6 years ago called streams64.exe, which I used to bulk unblock files I had downloaded via browser, because it bothered the hell out of me that every time I double-clicked a downloaded file, it would ask me to hit "Open" again - not to be confused with the User Account Control prompt. As always, thank you, ThioJoe!
I believe I figured the Anaheim thing out. About Anaheim Anaheim is designed to be used freely across the internet by web browsers on desktop computers, laptops and mobile devices. And Windows UI itself uses the Anaheim font
I came across these streams some 10-15 years ago when transferring video files from DVR to PC (DRM wasn't a big thing yet back then). The programs on TV often contained colons, and the non-Windows DVR just included them in the filenames. And the shoddy transfer program wouldn't bother converting the filenames to be Windows-compliant. If I forgot to change the filename before transfer, instead of 6-GB file named _some recorded:program.vid_ there would be a 0-byte size file named _some recorded_ . I quickly found instructions how to copy the contents of the file's side stream to the main stream in a new file, but it was annoying because the computer I used for that was an originally very low end laptop that was already old at that point and had a slow HDD, and copying large files to the same drive took for ages (because the HDD multitasked poorly). But often there was no choice, as I had set the transfers to be _moves_ , meaning the file would have been deleted from the device when I'd realize my mistake on the computer. (As hindsight, that wasn't even bad. Nowadays DRM often outright prevents keeping the recordings.)
Man, I'm so old school. Um,, since the 80's. Lol. So I still work mainly in C-pmpt. I Like that Joe demonstrates power user techniques. Win OS has many developers hacks and tweaks available to power users. This is a rare comprehensive tutorial. I would like to see more about H-key use. Kudos bro.!!!
I knew that ADS exist, but now it became much clearer, thanks! Everytime you copy a file to FAT32 and the system asks if you want to continue to copy a file without it's "properties" you are gently reminded they exist ;)
I actually use these all the time as temporary files when programming. I normally remove them because they contain hashed sensitive data but that’s what I use them for.
I thought I was super clever back in the day making a batch file "pw manager" with this approach. Hell, it would still throw off most people today but certainly not the most secure method on the planet lol.
Data streams have been the way the NTFS file system has worked since the second version - mid '90s . Every file has at least one stream - $Data. Most decent AV software has been scanning alternative straems since not long after NT4 hit the streets. This really is nothing new, they aren't hidden file - it's just the way NTFS works.
Trivial but important: add/change/remove an alternate data stream will also change the modification date/time. When programming a script or whatever you should restore the original date/time of the file afterwards.
Sysinternals suite has a tool for alternate data streams too - "streams". By the way, the eset antivirus used to write calculated checksums of files into the ads of the file, thus touching every file it scanned. Which drove any other antivirus or malware scanner nuts which then reported the whole disk as infected with malware.
point to note : to unblock files in a whole folder, there's a one line powershell command for it Get-ChildItem -Path "path to the folder" -Recurse | Unblock-File Get-ChildItem lists files / folder, -Path tells it where, and -Recurse tells it to navigate the subfolders also Unblock-File is pretty self-explanatory It's pretty usefull when grabbing powershell modules / scripts, like... the VMWare "powercli" suite, as without this command you won't be able to use any of them
I wonder if this could be related to something I've encountered recently, that is, PHP files infected with malware; when I open them with Nano, I don't see the malicious part of the code, but when I use Vim or Notepad++ via FTP, the malicious code is there. 🤔
Nano is a linux thing, so I suspect you're using linux, which would probably mean you're using ext*, btrfs, xfs or zfs, none of which, to my knowledge, support alternate data streams. What's imo more likely is that the malware replaced nano, or that for some reason nano has an different inode pointing to the safe file, while the ftp and php servers have one pointing to the malicious one, caching may also be an issue. The way linux usually works is that it has inodes, which point to files, and if a file is replaced, it is actually copied to a new space, and when a new call to open the file is made the new pointer will be given, but the old file is still there while there's a program using it, and so if the file isn't reopened completely, you may have a discrepancy between the two.
Great video, I already know about those "Super Hidden" but bet many don't. Even tho it's a year old still nice to watch, dunno how I missed this upload a year ago..
"Anaheim" may be related to Chromium-based Edge (which may be used for services behind the scenes by Windows even when you're using a different browser), and in conjunction with SmartScreen, I would guess that it's another marker to tell the OS that the file was handled by Edge? Mostly guessing after seeing that "Anaheim" is/was a codename for Edge; take this with a big grain of salt
I came to the same conclusion after doing a bit of googling. Anaheim was indeed the codename for Microsoft Edge, so Anaheim probably marks the file as being downloaded using Microsoft Edge.
Alternate data streams can also exist in other OS. You CAN have an alternate data stream attached to a folder, ANY folder, even the root folder of a drive.
Though this may have its issues, for example, any non-Windows user would be unable to solve the puzzle. He also mentioned that ADSs would vanish when uploaded to the Internet, so that would force said ARGs to have you run a script or program for the data to appear
@@martinus_mars WinRAR has an option to keep them in when compressing and extracting files. So while it might technically require running a program for it to work, it's at least a program that the vast majority of people are going to have.
@@Sypaka Not necessarily. Say you were using a game as a front for your ARG. Assuming that it isn't being distributed though Steam, you'd probably distribute it as a compressed file. You can then have the alternate data streams hidden within game files, allowing for the ARG to work. I already have plans for doing this exact thing myself (although, before hearing about alternate data streams, I was planning on distributing the information through the spectrograms of the game's ost). So yeah, not entirely pointless.
@@alexbubble6952 Depends on the game engine, but I have yet to see a game which has a filestructure, which allows files WITH their ADS to be stored within. However, you could make the game in a way to check, if the current folder is writeable, then extract a file from the gamefiles and move it under a gamefile as an ADS. But be aware, you may get flagged as malicious in heuristics. There is also a shit ton more to do on files itself. I made a file, which opens perfectly as JPG, ZIP, RAR, BAT and HTML all with different contents. Hilarious.
11:37 Anaheim is the city where Disney Land is located. It probably is related to this: "Microsoft rolled out the latest version of its Edge browser, Edge 79, on January 15, 2020. This update, codenamed ‘Project Anaheim’, is a landmark shift for Microsoft, from the EdgeHTML engine to the Chromium engine."
ADS 'files' show up on nearly every virus and mal-ware scanner. It used to be very common for those files to hide themselves this way behind known OS files. Fun fact this is how some programs used to check if a file was downloaded. I think IE used to mark downloaded files with a simple ADS mark. Fun fact 2. You can cleanse a file from ADS by moving it to a FAT32 USB drive and back since that file archetechture system doesn't support ADS and only the primay file will be saved.
Thanks for the exciting news - I truly had no idea. The years when DOS was my playground are long gone and I was so embarassed with missing out for so long about all the might and wonder of the powershell. Btw... although you just ask for a datastream "blah" the system suggests a .txt-file in 05:16. Why is that? Is it always this way when you enter no extension?
In the past you could also have different versions if the same picture as different streams. E.g.background pictures in different resolution or icons of different size. OS/2 did this with HPFS. There would be many other possibilities. But it will probably not happen anymore.
Dude, I've known this back when XP was still relevant, and every now and then, someone would hand me their USB, so that I could run "attrib -r -a -s -h /s /d" on it to unhide all super hidden files, and folders. 😅
6:22 I'd be wary of checking for an alternate data stream merely by file size versus disk space used. What if it's a 1GB file that uses 1.01 GB on disk? There could be a small alternate stream storing some info along with the 1GB of normal file data. Not a problem if your only concern is disk space, but for covert data even 1K might be a problem.
not only that, you could have a 1 byte file, that will show 4kb as size on disk because the partition is formmatted with that sector size, and it won't have any streams, its just how disks work
@@marsovac Yes, that is because NTFS does not allow mpre than one file per sector. Some file systems, e. g. Ext4 and Btrfs, theoretically allow inodes which contain more than one file. However, these files must share the same owner, group, creation and modification time.
@@pi_xiok yes, it depends on the filesystem, but probably there are quirks with those that support more than one file per sector as well. Alignment and padding come to mind to alleviate SMR/CMR differences in reading and writing capabiltiies. My point is that even without streams the method of comparing file size with size on disk is already faulty in concept since it has too many assumptions.
@@marsovac It can even be the other way around. Sparse files (often used for VM images) and compressed files (NTFS allows selective compression) take less space on the disk. NTFS junctions (hardlinks) take only one sector, which is usually 4 KB.
There are certain files you can't run at all on your machine even after disabling security settings. There is ONE specific setting in internet settings you would have to tick in advanced settings to allow the file to run.
@@ThioJoe Maybe is something related to the download of files or a presetup for edge related to the data streams. Remember that Microsoft integrate Ie at os level... maybe is still something of that under the hood. Greetings from Mexico...
@unsubtract The classic way Unix/Linux hides files is by starting their names with a dot. But use of this convention must be understood by individual programs. There's no way to tag other files to a main file such that all the files get copied or removed with the main file, unless smart versions of, say, cp, rm, and mv were created. One can view this as either a lot of trouble or a wonderful boon. But for a file to carry a hidden burden of "life altering" content could arguably be seen as unfriendly. Being of older school, I want it to be explicit.
@@SeekingTheLoveThatGodMeans7648 Yeah, dotfiles aren't attributes of a file system, they're just files like any other. Like you said, them being hidden by default just convention done by the ecosystem and is very very easily toggleable. You can copy dotfiles to a Windows system as normal. No such feature that is similar to forks/streams are present on Linux file systems like ext4 and Btrfs. There's just the file content itself, and then the metadata like uid/gid, inode, access/modify times, and the chattr ones which are stored/exposed by the file system. That's it.
ThioJoe you done it again you slick son of a gun! I'm a cyber sec major and I must say I've learned a lot of things about windows from you and not school lol currently taking windows workstation class but I've did a different one last semester
Malicious files that absorb disk space should be visible in the screen of a disk defragmenter that displays the clusters. I have detected certain small areas that are identified as areas occupied by 'system data'.
The most important thing I got from this video, is that there is an alternative to WinDirStat. I wasn't looking for an alternative, but now I can use the better alternative whenever I need it.
except those processes someone with malicious intent has hidden from the process listings which are also holding open unlinked files, those unlinked files still being written but not appearing is directory listings ...
That was used by Internet Explorer back then to store the bookmark url's favorite icon, also used by Windows Explorer to store summary data that user may describe for a file.
Well. First you have to actually execute the payload from them. But if your AV can't figure out how to read them or deal with them as they're used it's probably not worth using.
There are some "Super Hiden" Files, you can even locate them with the File Explorer: The Jumplist-Folders an Files are Super Hiden Files. They are located in the User profile in the path C:\Users\Name\AppData\Roaming\Microsoft\Windows\Recent\AutomaticDestinations and C:\Users\Name\AppData\Roaming\Microsoft\Windows\Recent\CustomDestinations. If you type in the path in the address bar in Windows Explorer, then you can see the Jumplist-Files of each Application of your Windows PC.
10:03 Damn any downloaded file has a tag with the source url ?! Do you know how often i did had to dig deep into the internet to look up the origin of the file ? That would have come in sooooo handy a couple of times !!! Thank you!!!
I have used ADS for over 20 years now to set the checksum of the file in there. I did this to check for disk errors. You can also use it to check whether you have a virus which encrypts your files (check your backups using a DIFFERENT computer to detect those).
Thanks. Now I know where to hide my "Homework" folder
Boku ..
No.......
...
@@azideiaman pico
no
@@BootlegGremlin hero academia 😀
Can I copy your homework?
I learned about Streams in 1998 in a security training class. They can be used to hide any file, even malicious files. anti virus scanners don't check for files in streams.
I believe antivirus programs do now
Umm hello
@@ThioJoe and even if they didn't, they will after seeing this 😆
You said don’t check files in stream why? So these hidden files should I delete,I just bought the surface go 13 inches. The only thing I can’t find is the service key it’s a long # where do I find it? I thank you your so informative & very intelligent person. You are AWESOME
username checks out
I made my living in IT for over 25 years. Started out with standard desktop break-->fix work and worked my way up to Enterprise WAN management. I've also done some scripting and programming. I would easily fit into the "expert" category (though I never liked calling myself that) and was often referred to as such - if someone was told to call the expert, they'd be calling me. In all those years, I never even heard of alternate data streams in Windows.
Like they say, "you learn something new every day." My hat comes off to you for even finding this feature and then explaining it, well, "expertly." Well done!
I've worked in IT for about ten years and know stuff about Windows that even some of the true graybeards around here didn't know... I also didn't know about this. Windows is such an insanely deep rabbit hole.
I've worked in it for as long as OP and DID know about ADS's. They have some practical uses but mostly exist as optional features unused by most users and software. The fact you can use traditional CLI commands to manipulate them is a little remarkable. I thought that would have been handled mostly through Powershell. This is a fascinating presentation.
Grey beard here, knew all about ADS and played with it, hiding stuff in there, cleaning virus out of that space, etc. More utilized under Apple products, but I was doing Windows.
You'll find ADS in your saved URLs, zone information is saved in there, even to this day. I usually clean up poisoned URLs so they are normal 2 digit size, not 3 digits in size.
I'm sure there's other things I don't know but I've seemed to have forgotten it. 😊
The term "superhidden" is itself in registry used for "protected system files". The NTFS data streams are not separate files, rather attributes.
Even if they aren't technically "files", practically speaking they are, and the fact they are so hidden is a considerable security threat.
@@fllthdcrb When was the last time you "on practice" accessed a data stream as a file? Yes, never, thank you.
"Security threat" - that's adorable. OS is aware of them, file system is aware of them, antiviruses are aware of them, IT pros are aware of them... the mere fact that YOU aren't, does not make 'em a security threat lol. It's a file system feature which may be used by malware - just like any other OS feature - e.g. you know, malware is usually stored in (drumroll) files! This does not make files a security threat.
@@fllthdcrb next time dont speak your ignorance.
@@fllthdcrb pwnd
@@hackdesigner >When was the last time you "on practice" accessed a data stream as a file?
That is literally how you access data streams.
>Yes, never, thank you.
You know when you access a file? Psyche, you're accessing the file's default data stream.
>"Security threat" - that's adorable.
About half of google search results are security blogs.
You can also use WSL to create a folder NUL or use parted magic to create folder with forbidden characters or use UNC paths to create folders ending with space alongside a folder not ending with space.
Hmm intradesting 🤔
Me wondering about every word he said be like
Looks like someone learned one or two things from Fly Tech :))
You can also use ALT + 0160 to create a folder without a name; 😈 My brother's best friend (RIP) taught me that back in 2006. 🤩
@@ThioJoe you should do it probably
As a noobie data-hoarder, the Zone Identifier seems extremely useful.
It's a shame that it's hidden away like that, I really like the idea of files keeping track of their origin.
On the contrary, I was always annoyed by this feature, so at some point I looked for a way to disable it and now I always disable it. I've known about alternative streams for a long time, but I didn't know that the zone identifiers are stored in them
Usually when I hear of "super hidden" files, it has to do with a completely separate concept from Alternative Data Streams (available as far back as Windows NT) which are the desktop ini files. These alter how Explorer shows (or hides) the current folder (and sometimes even its parent folder!) in every Windows version since Win 95. The most notable example is the recycle bin. You can see its true structure inside cmd or winfile (if available) but not within Windows Explorer.
you can see the real structure in explorer when opening via the hidden folder at C:\
@@the_lenny1 Still doesn't show the real data, just the interpreted data from the $R and $I files.
@Evi1 M4chine On older versions of Windows (95 for instance), they'd refuse to display certain folders in explorer no matter which UI options were toggled. You'd have to launch explorer from a command window to get around it (or just use the command window itself or the old File Manager.) Somewhere along the way, they gave more sane options to display these, at the expense of hiding other stuff (including alternative data streams.)
you teach me something pretty much every time i upload. hopefully one day i can keep this knowledge heading downstream and help other people. thank you!
I work in IT and did not even know this. I also always wondered how the OS would know a file came from the internet. I thought it might be some database or registry somewhere but it was odd because when I copy the file to another computer this would stay with the file...now I know. Thank you.
When we first launched NTFS we had a whole section in our 5-day training course, dedicated to streams & how to access them. The vision of the dev team was quite vast. 1. They'd let you extend the NTFS properties, so you could add your own custom tags needed to support an advanced Document management system. each file became the equivalent of a row in a doc database.
2. You could do version control. Keeping the delta between the version as streams. 3. You could do translations. The doc could contain streams for French, Spanish, English etc. & your application opens what it likes.
4. Similar to (1) Audio & Video could be tagged with production attributes commonly used in radio or TV production.
Unsurprisingly it never took off. Most Architects & developers don't get close enough to the API's to really understand how extensible & powerful the Microsoft products really are. So they don't think to use them, or they design some complex system to do the same thing the product does better out-of-the-box. I'm not saying that is their fault. There is only so many hours you can devote to really learning a product.
The irony is this indexing technology is much-needed but not yet user-friendly enough.
Another great thing are Junctions. I used them on a Win2K fileserver to build a nice tree from different partitions and shares. 20yrs later they are still not editable by UI but implemented in CLI commands.
I knew about this when I checked out the alternate streams option in 7zip, in which I found the zone.identifier file, and now you have confirmed those things do exist
Alternate data streams (in theory) could also be used for version control and document history. I'm surprised this isn't in wider use.
Metadata can be attached more easily is my guess; most programers dont want to bother with adding additional files to a files attributes to attach them to said file.
@@Atsumari That and the fact that this is Windows-specific.
More likely is the fact that it hides the data by design, kind of defeats the point of trying to keep track of version history if you obfuscate the data!
@@Fifury161 Well the idea would be that the application that uses ADS to store version info would also have the ability to scan ADS to retrieve it. The obfuscation in this case is actually useful because (if combined with encryption) it can prevent people from tampering with the versions.
@@bwcbiz technically you could use it on an NTFS-based Linux install
My favorite trick that blew my mind was when I learned you could unzip Microsoft office files and see all the hidden plaintext xml files ☺️
Omg, when i was kid i used "open with" on every file for lot of programs...
@@sophiacristina Right?!?! oh gosh, I wasn't that intelligent. I found out how to enable dictation and then plugged in my radio to the aux input and let the poor (according to today's standards) dictation engine type away seemingly random phrases and make me a "story" lol. I did find a Java page and the hidden Media songs but I had no idea what to do with those. Microsoft put so many surprises into these wonderful machines
@@SquirrelTheorist Hey, in fact that looks pretty smart and fun... :p
@Sophia Cristina XD Aw thank you! It was super fun. In a way I would totally do it again if the dictation programs today would type words out if it heard random static and music. That's a great idea, I think I'm going to give it another shot on this computer. Thank you Sophia! :D
only for DOCX and the likes..
This info was shown in a German "Computer Bild" PC magazine around 1995, when I was still in Highschool. I'd never have thought that this "feature" would be carried on until today, as it didn't make any sense back then - and it does not make any sense today (except another target of viruses).
Streams came with the NTFS file system.
Nice to know the access is so simple!
I used to use a program that made fake "bad sectors" on the disk, and then wrote to those.
Only a few advanced forensics programs can find them, and if encrypted in a certain way, even that can be ofuscated.
Worked on all operating systems.
And the name was ... Any more info on this.
Interesting...
I wouldn't put too much stock in the idea that this is ignored by forensics software though. But it could be a useful way to hide personal info in a way obtuse enough that malicious actors would fail to find it. Basically, don't use it to do illegal things, but it might be useful for stashing say, a crypto wallet's key or keypairs.
@@stevem1097 I think I figured out why there's no software names - every time i put any in, the comment gets nuked.
There's two things you want to do to accomplish this. First, manually and arbitrarily mark bad sectors, then read/write to sectors using low level disk access. If you google both topics you should find a few solutions pretty easy (the ones I can't seem to share with you for some reason)
Be CAREFUL if you mess with this. You're not working at a level where partitions and files exist anymore. You can seriously mess up your system.
@@stevem1097 I would also suggest that you think about process first, software second, in the future - considering how the comment reads, the actual piece of software, if the original commenter even remembers the name, might very well not exist or be deprecated by now. But if you know what you're trying to accomplish and can translate it into process, you can then attack each step of the problem individually
My "homework" folder is still hidden better ...
As in its not even existent
OS/2 has a nice use for this. It uses REXX as a scripting language (like Windows uses PowerShell), but that is a compiled language. It will be compiled automatically when you start a REXX file, but that could takes several seconds back in the day. To speed this up, the compiled version i saved in a data stream. So starting a REXX program is only slow the very first time you run it.
Suddenly I remembered that I mentally blocked out my experiences with OS2 Warp (and Lotus Notes)
One problem with Windows, and I suspect OS/2 as well is that filesystem features are too closely tied with one particular volume format, namely NTFS in the one case, HPFS in the other. On Linux, you have a VFS layer in the kernel which supports the use of a whole range of filesystems, including compatibility drivers for the old ones from DOS, Windows, and OS/2.
The trouble with the “data stream” concept is that it does too much, and yet too little. It’s like a directory which can only contain files to one level, and with limited names: why not replace it with an _actual_ directory?
@@lawrencedoliveiro9104 That's what OSX does. (On the other hand, you cannot install OSX on FAT32, so that workaround is useless...)
But the issue is something else: You cannot innovate if you want to stay compatible with the oldest technology ever made. At some point you want your operating system to have a filesystem that can do more than FAT8 with 6.3 character ASCII filenames, 8MB file size limit and no directories. And when adding more and more features into your filesystem (last access time? check. last modification time? check. creation time? whatever. owner? go on. group? ... acls?) you will come to the point where it stops making sense to add specific fields for each and every possible data value. And at that point the only logical solution is to have a generic structure, Extended Attributes in OS/2, File Streams in Windows. Both are basically simple key-value lists.
The real interoperability issue here is that there is no way to store those on FAT volumes. So every operating system finds their own way to store them, and every one does it differently. In theory all operating systems could use the corresponding storage system of any filesystem like their own, if they had drivers for those. But they generally don't, so we end up with all that garbage on FAT volumes.
NTFS is showing its age. Windows desperately needs a more modern filesystem. But NTFS is too heavily baked into Windows to be easy to replace.
That's a good use of these, for non-critical information or as a cache, since these alternate data streams generally aren't preserved when sending the file across the Internet.
When the input file contains a [Ctrl]+"Z" character the TYPE command stops there with displaying text. Typically a remnant of the MS-DOS days. I suggest using the COPY /B variant to add an alternative data stream.
I used to clear out the viral hidden junk after the Ctrl Z symbol in text files.
Then how did you read it?
@@bruhuk_obama Using apps that ignore the Ctrl-Z character
"Anaheim" in the SmartScreen files? Interesting. The codename for "new Edge" (the Chromium based one) is "Anaheim".
ANAHEIM is also one NSA project. Coincidence?
Well shit, this is actually something I was concerned about. I was wondering why there was so little space on my pc when I've barely downloaded anything. Maybe I accidentally downloaded a virus.
There are many other possible causes though, use WinDirStat or WizTree to figure out where all your storage has gone
@@ThioJoe thanks for the advice!
that space is most likely held hostage by windows itself. If you have hibernation enabled, there'll be a hibernation file about the size of your system memory where windows dumps its RAM contents before it goes to sleep. So on a system with 16GB of RAM that's 16+GB of your SSD gone. If you haven't configured your system memory manually then windows will create a swap file that can be anywhere from 1GB to 150% of your machine's ram size. A 5GB swap file is pretty normal with memory management set to auto but it can grow much larger than that. My workstation has 128GB of RAM and for normal usage a swap file isn't necessary at all since memory usage is only 10 or maybe 20% ... but left unchecked windows wont hesitate to create a 40-60 GB swap file ... which is of course of no actual use and only slows my system down while eating away at my SSD. By default windows also reserves a considerable portion of disk space for windows update downloads and update backups/ update roll-backs. I think after my last fresh install it was already about 8GB or so. System restore points also take up a lot of space if you create them regularly / before every software installation - which is a good thing to do, but managing restore points and deleting old ones will free up maaany GB instantly. No need to keep more than the latest two restore points really. It's also always good to check the systems temp folder(s) once in a while ... sometimes install packages aren't deleted on shut down for some reason and then linger for quite a while. WIndows by default will also use it's oh so great prefetch and superfetch functions to pre-load/ cache data of frequently used software to speed up loading times. Obviously useless with Sata and NVMe SSD system drives.
Windows is a pain in flank these days. On a vanilla W10/11 install with all default settings losing 50-100GB of disk capacity to Microsoft's nonsense is normal.
This is honestly SUPER useful for hiding watermarks in any art files that are sold, such as 3D models. Makes it easy to prove ownership if someone tries reposting it as their own.
Great call on WizTree. When we used Kaspersky at work several years ago some log file it created used up all the space on my hard drive. WizTree was the first utility I ran that found what had used the space.
We thankfully no longer use Kaspersky, but I bought a personal license for WizTree.
Also the ASV utility seems to be pretty useful too.
Great job on a topic relatively few windows users know about. Those who are security focused will know about this, but the tools you present to find data streams are good for general users. Well done.
Heh, extracting data from Mac files on an NT 4.0 server was exactly how I learned about streams. I hacked together a few utilities & menu items so I can quickly see where downloaded files originally came from. Lots of cool things, and lots of programs like Firefox, Notepad++ etc. can get at the streams if you know they exist. I've had way too much fun messing around with them.
I went into this video thinking I wouldn't learn much as I was already familiar with ADS, but you taught me a lot! Very thorough and informative video! Thanks a lot man!
This feature seems like an excellent way to add some difficulty to ARGs.
It's an NTFS/ReFS/UDF attribute, so it wouldn't work in most cases as this information isn't converted/preserved when copying the file into a zip archive, website, non Microsoft file system, etc.
If the ARG consists of a program that install stuff on the computer, then sure this could work as the program can modify/add the alternate data stream attributes but if say the ARG had an image that you'd download off the internet, you can't hide additional information in it. You can only store the contents of the file itself and the file name (not even metadata like permissions is possible).
Often a lot of programs like file archivers will retrieve & convert attributes like permissions & access timestamps from the FS to store it in its archive format and then convert & apply it on other systems when a file is extracted, like Windows permissions being read and then stored in the zip archive format which when sent to and extracted on a Linux system, the file archiver will set the extracted file's permissions & access timestamps with the data stored in the archive format during extraction. Alternate data streams are not one of these attributes that most programs preserve as they're pretty specific to Windows systems, unlike more universal things like permissions and access timestamps.
@@ecksdee8072 OK Jesus Christ that is way too complicated for me to understand
@@SusuLakuProductions ok
A platform independent alternative is hiding a zip file insize an image file. Note that this will add to the reported file size and depending on how big the zip file is compared to the expected image size from its quality and all, some people may catch the discrepancy and look into it.
Participated in an event where they did exactly that. I believe the practice is called Steganography.
@@Vixel4076 yeah, steganography basically is "hiding a message within another message or object" though depending on which angle you look at it, if you look from the file angle, it is steganography, but if you look from the image side some may not consider it steganography because if you screenshot or convert it losslessly, the hidden file disappears.
A decent way to hide info is also within docx files, which are basically zip files with some structure, and can be made fairly large if they contain pictures, so a few hundred kb can pass fairly easily.
For larger files, a lot of videogames use a compressed file, sometimes encrypted (but since the game needs to open it, it is possible to get the key), so you can hide a couple gigabytes of files in there.
11:32 Anaheim is the codename for SmartScreen.
Keep up the good work As a avid Linux user I decided to jump into windows and tackle it and learn as much as I can learn and you’re really a lifesaver for me thank you for the video’s
By the way, Thio, I'm about to use your YTSpammerPurge tool right now, because I'm starting to get really annoyed by spammy comments, so I figured I'd do some "janitor" work in my free time for some RUclipsrs I admire. Great job, bro! 👏🏻
See, this is why I always end up going back to watch a video even when the title isn't "clickbait-y" enough, and I feel so damn lucky that I have the tolerance to sit through anything, and in the end, it's very rewarding 99% of the time. I wonder if this alternate data stream thingie is related to an executable I used 6 years ago called streams64.exe, which I used to bulk unblock files I had downloaded via browser, because it bothered the hell out of me that every time I double-clicked a downloaded file, it would ask me to hit "Open" again - not to be confused with the User Account Control prompt. As always, thank you, ThioJoe!
I wrote all this before finishing the video. LMAO Man, you just made me realize that I have the memory of an elephant, bro. 😂🤣
cool story bro
I believe I figured the Anaheim thing out. About Anaheim
Anaheim is designed to be used freely across the internet by web browsers on desktop computers, laptops and mobile devices. And Windows UI itself uses the Anaheim font
I came across these streams some 10-15 years ago when transferring video files from DVR to PC (DRM wasn't a big thing yet back then). The programs on TV often contained colons, and the non-Windows DVR just included them in the filenames. And the shoddy transfer program wouldn't bother converting the filenames to be Windows-compliant. If I forgot to change the filename before transfer, instead of 6-GB file named _some recorded:program.vid_ there would be a 0-byte size file named _some recorded_ . I quickly found instructions how to copy the contents of the file's side stream to the main stream in a new file, but it was annoying because the computer I used for that was an originally very low end laptop that was already old at that point and had a slow HDD, and copying large files to the same drive took for ages (because the HDD multitasked poorly). But often there was no choice, as I had set the transfers to be _moves_ , meaning the file would have been deleted from the device when I'd realize my mistake on the computer.
(As hindsight, that wasn't even bad. Nowadays DRM often outright prevents keeping the recordings.)
Interesting story, thanks.
@Evi1 M4chine Well, it's not exactly trivial.
The NirSoft person is insane. I always thought it was a team of people. Super useful tools!
Edit: and nice video! Didn't know about this at all.
He's a friggin hero
Powershell has stream related commands, which can help in creating, finding and deleting alternate data streams.
Man, I'm so old school. Um,, since the 80's. Lol. So I still work mainly in C-pmpt. I Like that Joe demonstrates power user techniques. Win OS has many developers hacks and tweaks available to power users. This is a rare comprehensive tutorial. I would like to see more about H-key use.
Kudos bro.!!!
Nice! This is really cool, thx ThioJoe!
I knew that ADS exist, but now it became much clearer, thanks! Everytime you copy a file to FAT32 and the system asks if you want to continue to copy a file without it's "properties" you are gently reminded they exist ;)
I actually use these all the time as temporary files when programming. I normally remove them because they contain hashed sensitive data but that’s what I use them for.
I thought I was super clever back in the day making a batch file "pw manager" with this approach. Hell, it would still throw off most people today but certainly not the most secure method on the planet lol.
Data streams have been the way the NTFS file system has worked since the second version - mid '90s . Every file has at least one stream - $Data. Most decent AV software has been scanning alternative straems since not long after NT4 hit the streets. This really is nothing new, they aren't hidden file - it's just the way NTFS works.
The alternate data streams would be hidden if they are encrypted. You can take it a step further but that would be telling :)
Trivial but important: add/change/remove an alternate data stream will also change the modification date/time. When programming a script or whatever you should restore the original date/time of the file afterwards.
Sysinternals suite has a tool for alternate data streams too - "streams".
By the way, the eset antivirus used to write calculated checksums of files into the ads of the file, thus touching every file it scanned. Which drove any other antivirus or malware scanner nuts which then reported the whole disk as infected with malware.
I tried opening streams as admin but nothing happened after the confirmation popup. 🤔
@@zmbdog this platform keeps removing my comments, sorry, dude, I'm done here.
@@furzkram That's ok. I can see your previous comment in my notifications still. Thanks
point to note : to unblock files in a whole folder, there's a one line powershell command for it
Get-ChildItem -Path "path to the folder" -Recurse | Unblock-File
Get-ChildItem lists files / folder, -Path tells it where, and -Recurse tells it to navigate the subfolders also
Unblock-File is pretty self-explanatory
It's pretty usefull when grabbing powershell modules / scripts, like... the VMWare "powercli" suite, as without this command you won't be able to use any of them
I wonder if this could be related to something I've encountered recently, that is, PHP files infected with malware; when I open them with Nano, I don't see the malicious part of the code, but when I use Vim or Notepad++ via FTP, the malicious code is there. 🤔
Nano is a linux thing, so I suspect you're using linux, which would probably mean you're using ext*, btrfs, xfs or zfs, none of which, to my knowledge, support alternate data streams.
What's imo more likely is that the malware replaced nano, or that for some reason nano has an different inode pointing to the safe file, while the ftp and php servers have one pointing to the malicious one, caching may also be an issue.
The way linux usually works is that it has inodes, which point to files, and if a file is replaced, it is actually copied to a new space, and when a new call to open the file is made the new pointer will be given, but the old file is still there while there's a program using it, and so if the file isn't reopened completely, you may have a discrepancy between the two.
Great video, I already know about those "Super Hidden" but bet many don't.
Even tho it's a year old still nice to watch, dunno how I missed this upload a year ago..
"Anaheim" may be related to Chromium-based Edge (which may be used for services behind the scenes by Windows even when you're using a different browser), and in conjunction with SmartScreen, I would guess that it's another marker to tell the OS that the file was handled by Edge?
Mostly guessing after seeing that "Anaheim" is/was a codename for Edge; take this with a big grain of salt
I came to the same conclusion after doing a bit of googling. Anaheim was indeed the codename for Microsoft Edge, so Anaheim probably marks the file as being downloaded using Microsoft Edge.
Thank you for telling me Wiztree. It baffles me how much faster it loads the interface. It even has dark mode, and a great UI design.
I’m a next level Windows expert and I don’t even know what this is and I know basically everything in Windows and how it works
You weren’t lying
That Nirsoft guy is a super hero. I wish he got more recognition.
I remember learning this from another youtuber. It was Enderman I think.
FlyTech Videos has a video for this
Yea not surprised, he makes lots of good videos about lesser known windows stuff
Alternate data streams can also exist in other OS.
You CAN have an alternate data stream attached to a folder, ANY folder, even the root folder of a drive.
Interesting video.. I wonder if ARGs will make use of Alternate Data Streams in the future.
Though this may have its issues, for example, any non-Windows user would be unable to solve the puzzle. He also mentioned that ADSs would vanish when uploaded to the Internet, so that would force said ARGs to have you run a script or program for the data to appear
@@martinus_mars WinRAR has an option to keep them in when compressing and extracting files. So while it might technically require running a program for it to work, it's at least a program that the vast majority of people are going to have.
Nope, because any Browser overwrites the ADS and you cannot even upload ADS.
You can kinda embed them in rar/7zip, but pointless imo.
@@Sypaka Not necessarily. Say you were using a game as a front for your ARG. Assuming that it isn't being distributed though Steam, you'd probably distribute it as a compressed file. You can then have the alternate data streams hidden within game files, allowing for the ARG to work. I already have plans for doing this exact thing myself (although, before hearing about alternate data streams, I was planning on distributing the information through the spectrograms of the game's ost). So yeah, not entirely pointless.
@@alexbubble6952 Depends on the game engine, but I have yet to see a game which has a filestructure, which allows files WITH their ADS to be stored within.
However, you could make the game in a way to check, if the current folder is writeable, then extract a file from the gamefiles and move it under a gamefile as an ADS. But be aware, you may get flagged as malicious in heuristics.
There is also a shit ton more to do on files itself. I made a file, which opens perfectly as JPG, ZIP, RAR, BAT and HTML all with different contents. Hilarious.
11:37 Anaheim is the city where Disney Land is located. It probably is related to this: "Microsoft rolled out the latest version of its Edge browser, Edge 79, on January 15, 2020. This update, codenamed ‘Project Anaheim’, is a landmark shift for Microsoft, from the EdgeHTML engine to the Chromium engine."
Very clearly presented and useful information. Thank you for sharing!
An alternate data stream is not a separate file. It is an additional “attribute” in NTFS
When can we expect the PC build video? Excited for it 🤞
This week some time
@@ThioJoe LFGGG
As a Linux user who looked into Windows' drive via Linux file manager, I knew the existence of all of these directory.
~I use Arch BTW~
Nice one. No irrelevant chatter just real information.
ADS 'files' show up on nearly every virus and mal-ware scanner. It used to be very common for those files to hide themselves this way behind known OS files.
Fun fact this is how some programs used to check if a file was downloaded. I think IE used to mark downloaded files with a simple ADS mark.
Fun fact 2. You can cleanse a file from ADS by moving it to a FAT32 USB drive and back since that file archetechture system doesn't support ADS and only the primay file will be saved.
Yup, ADS is exclusive to NTFS.
Thanks for the exciting news - I truly had no idea. The years when DOS was my playground are long gone and I was so embarassed with missing out for so long about all the might and wonder of the powershell. Btw... although you just ask for a datastream "blah" the system suggests a .txt-file in 05:16. Why is that? Is it always this way when you enter no extension?
Hm I'm not sure, I assume it is something Notepad just did by default for some reason since a filetype wasn't given.
In the past you could also have different versions if the same picture as different streams. E.g.background pictures in different resolution or icons of different size. OS/2 did this with HPFS. There would be many other possibilities. But it will probably not happen anymore.
As someone working as a software engineer your videos interesting and informative!
Dude, I've known this back when XP was still relevant, and every now and then, someone would hand me their USB, so that I could run "attrib -r -a -s -h /s /d" on it to unhide all super hidden files, and folders. 😅
The place to go when realizing that I know little about the magic box 📦. Thank you for all the great work!
6:22 I'd be wary of checking for an alternate data stream merely by file size versus disk space used. What if it's a 1GB file that uses 1.01 GB on disk? There could be a small alternate stream storing some info along with the 1GB of normal file data. Not a problem if your only concern is disk space, but for covert data even 1K might be a problem.
not only that, you could have a 1 byte file, that will show 4kb as size on disk because the partition is formmatted with that sector size, and it won't have any streams, its just how disks work
@@marsovac Yes, that is because NTFS does not allow mpre than one file per sector. Some file systems, e. g. Ext4 and Btrfs, theoretically allow inodes which contain more than one file. However, these files must share the same owner, group, creation and modification time.
@@pi_xiok yes, it depends on the filesystem, but probably there are quirks with those that support more than one file per sector as well. Alignment and padding come to mind to alleviate SMR/CMR differences in reading and writing capabiltiies. My point is that even without streams the method of comparing file size with size on disk is already faulty in concept since it has too many assumptions.
@@marsovac It can even be the other way around. Sparse files (often used for VM images) and compressed files (NTFS allows selective compression) take less space on the disk. NTFS junctions (hardlinks) take only one sector, which is usually 4 KB.
Alternate data streams - I learned of them at least 20 years ago.
i always read the ntfs as nfts
Thanks
Hacker mode activated. Thanks Thio!
Already planning to rewrite some of my proof of concept malware. To take advantage of this.
There are certain files you can't run at all on your machine even after disabling security settings. There is ONE specific setting in internet settings you would have to tick in advanced settings to allow the file to run.
Anaheim is the code project name of Edge Chrome version...
Hmm that could explain it, but i didnt use edge to download them 🤔
@@ThioJoe Maybe is something related to the download of files or a presetup for edge related to the data streams. Remember that Microsoft integrate Ie at os level... maybe is still something of that under the hood. Greetings from Mexico...
Nice! I always wondered why files I dragged from my Windows file system to my WSL file system carried that “ZoneIdentifier” bit
this is interesting. wonder if this will work on macOS/*NIX systems
They have "forks" instead of streams on apple file system, so theoretically yea
@unsubtract The classic way Unix/Linux hides files is by starting their names with a dot. But use of this convention must be understood by individual programs. There's no way to tag other files to a main file such that all the files get copied or removed with the main file, unless smart versions of, say, cp, rm, and mv were created.
One can view this as either a lot of trouble or a wonderful boon. But for a file to carry a hidden burden of "life altering" content could arguably be seen as unfriendly. Being of older school, I want it to be explicit.
@@SeekingTheLoveThatGodMeans7648 Yeah, dotfiles aren't attributes of a file system, they're just files like any other. Like you said, them being hidden by default just convention done by the ecosystem and is very very easily toggleable. You can copy dotfiles to a Windows system as normal.
No such feature that is similar to forks/streams are present on Linux file systems like ext4 and Btrfs. There's just the file content itself, and then the metadata like uid/gid, inode, access/modify times, and the chattr ones which are stored/exposed by the file system. That's it.
@@ecksdee8072 you are forgetting user_xattr which does allow some extra metadata to be stored with the file, though size limited on ext[4|3|2].
I wonder if you pack file into archive does this stream get copied or not?
Great fann ❣️💓
ThioJoe you done it again you slick son of a gun! I'm a cyber sec major and I must say I've learned a lot of things about windows from you and not school lol currently taking windows workstation class but I've did a different one last semester
Something that Joe missed is that you can actually enable super hidden files by using registry editor. I forget how but I’ll look into it
I’d be interested in knowing where this registry key is.
I suspect you mean desktop.ini-like hidden files and not alternate data streams
Wouldn't the TYPE command also be able to copy the contents _from_ an alternate data steam _to_ a regular file?
Let's take the moment to appreciate how much effort he puts into his content for us. Great job. ❤
Let's take the moment to spot how fake your 'likes hunting' comment is
@@-COBRA Facts
Malicious files that absorb disk space should be visible in the screen of a disk defragmenter that displays the clusters.
I have detected certain small areas that are identified as areas occupied by 'system data'.
Hi
The most important thing I got from this video, is that there is an alternative to WinDirStat. I wasn't looking for an alternative, but now I can use the better alternative whenever I need it.
Use Linux and nothing is hidden from you at all
Cope + WSL
except those processes someone with malicious intent has hidden from the process listings which are also holding open unlinked files, those unlinked files still being written but not appearing is directory listings ...
So good to see you mention wiztree! Love it more than windirstat and treesize
Pin me for no reason 😅
Edit: nvm 😢
Edit 2: OMG A HEART THATS CRAZYYYYYY
ThioJoe hearted your comment again Once you edit a comment, the ❤️ will disappear so be careful 🙂
@@_SJ oh phew ty
That was used by Internet Explorer back then to store the bookmark url's favorite icon,
also used by Windows Explorer to store summary data that user may describe for a file.
Earlyish
Anaheim Electronics was from Terminator (movie) I think
Those data streams, added for no good reason, are a big security risk. Payload can easily be a virus, trojan, or keylogger.
Well. First you have to actually execute the payload from them. But if your AV can't figure out how to read them or deal with them as they're used it's probably not worth using.
That was a REALLY INTERESTING & educational video. Thank you TJ,
There are some "Super Hiden" Files, you can even locate them with the File Explorer: The Jumplist-Folders an Files are Super Hiden Files. They are located in the User profile in the path C:\Users\Name\AppData\Roaming\Microsoft\Windows\Recent\AutomaticDestinations and C:\Users\Name\AppData\Roaming\Microsoft\Windows\Recent\CustomDestinations.
If you type in the path in the address bar in Windows Explorer, then you can see the Jumplist-Files of each Application of your Windows PC.
This guy know operating system in deep level.
Thanks for sharing us this kind of useful information.
Thanks Theo for the great info👍🏻.
this is one of the best tech videos i had seen on youtube !!!... thanks
p.s. you look alot like connor from detroit become human xD
10:03 Damn any downloaded file has a tag with the source url ?!
Do you know how often i did had to dig deep into the internet to look up the origin of the file ? That would have come in sooooo handy a couple of times !!!
Thank you!!!
From time to time you teach me something. Great video, thank you!
I have used ADS for over 20 years now to set the checksum of the file in there. I did this to check for disk errors.
You can also use it to check whether you have a virus which encrypts your files (check your backups using a DIFFERENT computer to detect those).
Excellent presentation, thank you!
What terminal are you using, the tabs, how are you getting those?
This has taken me back 30 years. We used a program called x-tree gold to hide files etc.