Where does the OS reside after an upgrade installation ?

This is just something I wondered over.

For example... if you take a traditional HDD and install Windows then I believe the OS will reside in a contiguous portion of the drive. So if you had Windows installed on a 100Gb partition and the OS took 20Gb, then the first 20Gb are 'taken'. You then install your own programs and user files, so lets say the first 30 or 40Gb are now taken.

Assuming I'm correct so far, what happens when you upgrade Windows ?

Does the original OS get replaced in full and so still reside in that first 20Gb or so ? (give or take of course, if the new one is smaller or larger then there will be gaps or it will move further on).

After the upgrade we have a 'windows.old' folder that contains the old OS in case we need to go back.

Where does that now live ? Is it the old 'first 20Gb' still in its original location and bundled up to delete, or has it been moved to the next chunk of free space ?

Hope that makes sense and isn't to daft a question. I just wondered ultimately where the 'new upgraded OS' lives i.e. is it now still at the start of the original partition having replaced/overwritten the old OS, or is 'windows.old' still living there with the new OS elsewhere on the disk.

The file system (usually NTFS for a partition containing the Windows OS) keeps track of where the files are physically written on the disk. The files don't have to be contiguous, one file can be spread out over several locations on the disk. This is called fragmentation. The files system reads all the fragments of the file it needs and then pieces them together.

When a file is deleted, the spaces on the disk where that file was originally written get marked as available for re-use. When a new data needs to be written, the filesystem just looks for available spaces where it can write to and writes the file to those empty spaces.

Think of the partition containing C: drive, for example, as a chalkboard or whiteboard. You write on a space on the chalkboard - you can write anywhere on the space designated as the surface of the chalkboard - but not go beyond the edges of it. That would be the partition on the disk. When you no longer need to keep what is written on the chalkboard, you erase that spot. If you want to write something else to that spot you can.

About the biggest difference is if you want to write the number 5,367.32 on the chalkboard, you pretty much have to write the numbers next to each other to know what order they are in. The files system does not need to do that. The files system can write each number anywhere on the board and in any order. It keeps track of which number is written where and what order they are supposed to be in. So when it needs the number again, it goes and gets the 5 from one spot, the first 3 from another spot, the 6 from another spot, etc...then puts them back in order to recreate the 5,367.32.

The folder structure you see on the drive, such as C:\Users\John\Documents has nothing to do with where the files are located on the partition. The folder structure is completely managed by the file system. It's a virtual structure, not a physical structure on the disk. FAT as in FAT32 means File Allocation Table. That's one method of maintaining the folder structure. NTFS uses an MFT - Master File Table.
Master File Table (Windows)

Thanks for the reply, you've gone to lot of trouble over that :)

I think I follow what you are saying over the way the OS keeps track of everything... and yet when you clean install Windows on fresh partition/drive it will (I think) be written continuously in the first available space. It wouldn't be spread out over all the available space... would it ? That's the crux of what I puzzle over. Does the upgrade install reside in what was free space after the original OS, or does shuffle things around so that it ends up similar to the original set up.

I know ultimately it doesn't matter as such, I was just curious As to the mechanics of it all.

(Lol, typing this on a tablet as I"m clean installing just now)

The clean install to a new partition should write the files in contiguous spaces at the front of the partition because there will be nothing but free space available for the files being written.

During an upgrade, I would assume that the original files get "moved" to the windows.old folder. Now moving files into different folders does not physically move them on the hard drive, so the original files would stay on the same physical spots on the hard drive - only the entries in the Master File Table get changed to reflect what folder those files should appear in.

When the upgrade files get written, they get written to whatever free space is available. If any files were deleted, then the empty spaces vacated by those deleted files will get written to first. Once those are all filled up, then the next files coming along would get written to contiguous empty space following the filled up space.

Think of it as a parking lot. The first 10 cars in get the first 10 empty spaces contiguous closest to the store. Car #2 leaves. Now the next set of cars come in. Car #11 takes the empty spot created when car #2 left. The cars following that will now start filling up the empty contiguous spaces starting with the empty space behind car #10.

Left click on the start icon and start typing Optimize. Defragment and optimize drives should appear at the top of the list. Open that app. For hard disk drives, a % fragmented will be shown. That tells you what percentage of data is spread all over the partition. Periodically Windows will go through and move all the files around so they occupy contiguous spaces - defragmenting. That's because a file occupying a contiguous space takes less time to read, so Windows tries to keep things organized in the best way for you.

NavyLCDR said:

During an upgrade, I would assume that the original files get "moved" to the windows.old folder.

I don't think this is exactly correct (although I agree with your general point).

During an upgrade the first thing that happens is Windows is renamed to Windows.old. Nothing moves at this point.

Then certain bits are copied from Windows.old to (the new) Windows directory. Specifically parts of the registry, some driver files and so on.

Assuming you then run disk cleanup and you have a traditional HDD you will end up with free space at the beginning (fastest) part of the drive. This will later get full of junk as it is the first bit of free space.

You would be better (in this case) to backup and restore the OS. Not only to reduce fragmentation but to put the bits you use most often on the faster part. People certainly did this in the past as reading the beginning of the disk (the outside) is faster than the inside as the whole lot revolves at a certain RPM and there is more inches per second.

When I first started dual-booting OSX and Windows I always installed Windows first so it would get the faster part of the disk (I'm unsure if it helped much but I did it anyway).

On a SSD it is not important and only a concern if your MST gets so big to as to require defragmentation which is generally unlikely to happen unless you create and destroy a huge number of of objects.

No need to speculate on where files get placed - this handy Sysinternals utility will tell you exactly where every
file on your NTFS volume is located: DiskView

kreemoweet said:

No need to speculate on where files get placed - this handy Sysinternals utility will tell you exactly where every
file on your NTFS volume is located: DiskView

Not really. If you have an SSD it will only see virtual clusters as told to it by the controller. Doesn't tell you anything interesting at all.

Mooly said:

For example... if you take a traditional HDD and install Windows then I believe the OS will reside in a contiguous portion of the drive. So if you had Windows installed on a 100Gb partition and the OS took 20Gb, then the first 20Gb are 'taken'.

While it's *possible* this could be the case, it's unlikely in practice. The NTFS filesystem puts files wherever it wants. While, it won't just randomly put stuff anywhere on disk, it's not really going to be a contiguous block either.

Just as one example, NTFS has a concept called an MFT or Master File Table, which stores information about every file, and typically points to where it exists on disk. However, for small files (typically under 4k in size) they can exist solely in the MFT entry, instead of taking additional sectors on disk for both the MFT entry and the file itself.

The point here is that the OS is free to put data wherever it wants, and there's no real way for you to predict where it will store things, except when you defrag, which deliberately moves files to specific locations.

NavyLCDR said:

The clean install to a new partition should write the files in contiguous spaces at the front of the partition because there will be nothing but free space available for the files being written.

During an upgrade, I would assume that the original files get "moved" to the windows.old folder. Now moving files into different folders does not physically move them on the hard drive, so the original files would stay on the same physical spots on the hard drive - only the entries in the Master File Table get changed to reflect what folder those files should appear in.

When the upgrade files get written, they get written to whatever free space is available. If any files were deleted, then the empty spaces vacated by those deleted files will get written to first. Once those are all filled up, then the next files coming along would get written to contiguous empty space following the filled up space.

Think of it as a parking lot. The first 10 cars in get the first 10 empty spaces contiguous closest to the store. Car #2 leaves. Now the next set of cars come in. Car #11 takes the empty spot created when car #2 left. The cars following that will now start filling up the empty contiguous spaces starting with the empty space behind car #10.

Left click on the start icon and start typing Optimize. Defragment and optimize drives should appear at the top of the list. Open that app. For hard disk drives, a % fragmented will be shown. That tells you what percentage of data is spread all over the partition. Periodically Windows will go through and move all the files around so they occupy contiguous spaces - defragmenting. That's because a file occupying a contiguous space takes less time to read, so Windows tries to keep things organized in the best way for you.

That makes sense, and I follow what you mean about nothing really changing, just the original files now be flagged as located in windows.old.

So that really in a way answers what I was originally wondering... that the original OS files stay put. If at some later date you delete them then you would end up with a chunk of free space at the 'start' of the partition.

I'm actually running an SSD so I don't get to see the fragmentation, and that's something I would have been interested to know.

lx07 said:

I don't think this is exactly correct (although I agree with your general point).

During an upgrade the first thing that happens is Windows is renamed to Windows.old. Nothing moves at this point.

Then certain bits are copied from Windows.old to (the new) Windows directory. Specifically parts of the registry, some driver files and so on.

Assuming you then run disk cleanup and you have a traditional HDD you will end up with free space at the beginning (fastest) part of the drive. This will later get full of junk as it is the first bit of free space.

You would be better (in this case) to backup and restore the OS. Not only to reduce fragmentation but to put the bits you use most often on the faster part. People certainly did this in the past as reading the beginning of the disk (the outside) is faster than the inside as the whole lot revolves at a certain RPM and there is more inches per second.

When I first started dual-booting OSX and Windows I always installed Windows first so it would get the faster part of the disk (I'm unsure if it helped much but I did it anyway).

On a SSD it is not important and only a concern if your MST gets so big to as to require defragmentation which is generally unlikely to happen unless you create and destroy a huge number of of objects.

So this confirms NavyLCDR's thoughts on what happens.

Would a restore reduce fragmentation ? I'm not so sure on that one tbh. A disk image as far as I know makes no distinction as to what the backed up sectors are, they are just block of data as far as the imaging program goes. I can see how you might be able to restore and 'move everything up' if there was a chunk of free space at the start of a disc although even that might be specific to the image made. If its a sector by sector then I would assume it would retain the exact structure. But yes, I see your point... and mostly we don't do sector by sector imaging.

lx07 said:

Not really. If you have an SSD it will only see virtual clusters as told to it by the controller. Doesn't tell you anything interesting at all.

I do have an SSD. That alone has been the biggest single jump in performance I have ever seen. Forget extra RAM and tweak this and tweak that. The SSD just flies. Its orders of magnitude quicker than traditional spinners.

Mystere said:

While it's *possible* this could be the case, it's unlikely in practice. The NTFS filesystem puts files wherever it wants. While, it won't just randomly put stuff anywhere on disk, it's not really going to be a contiguous block either.

Just as one example, NTFS has a concept called an MFT or Master File Table, which stores information about every file, and typically points to where it exists on disk. However, for small files (typically under 4k in size) they can exist solely in the MFT entry, instead of taking additional sectors on disk for both the MFT entry and the file itself.

The point here is that the OS is free to put data wherever it wants, and there's no real way for you to predict where it will store things, except when you defrag, which deliberately moves files to specific locations.

I never knew that small files could exist in that form. One of the programs I use has several thousand files of data, all very small.

Interesting, thanks.