2012 deduplication savings

So now that 2012 has been out for a while… im starting to see a bit more detail on certains features… in this case file de-duplication.

I applied file deduplication to a drive we use called our “SOE” drive, which contains all the MS products, pre-prepared packages for SCCM, VHD (and vhdx) libraries etc…. a reported dedup hit rate of 42%, but on the folder, showing 8.5GB on disk out of 357GB in the folder – quite impressive…. but also not 42%, so where’s the gap?

So, when i ran out of NAS space, i purchased a stand-alone 3TB drive in a server – for storage of “less important” data, that has already been compressed using a combination of 7zip and rar…

As deduplication doesnt work on compressed files, i started decompressing – to see which is better, compression or deduplication. (I realise this depends on file types etc, but this data was a mix of iso’s, documents, executables etc)

I had used approx 2.2TB out of 2.72TB available while the files were compressed.

Once many of the folders were decompressed, i was, according to the properties of the folder, storing 2.16TB of data and only using 151mb on disk….. thats quite impressive, but the properties of the disk told a different story – 7oGB free….

so where has all the space gone?

Utilising one of my favourite tools – treesize by jam software, i had a quick look…. under “System volume informationDeDup” there was now 2.6TB of data… thats right, the dedup folder was larger than the original data itself!

Going back to my SOE drive, i discovered the deup folder was 208GB…. so my real “saving” on the SOE drive was only around the 148GB mark, not the 340-ish Gb the folder properties showed – still a good saving – but unfortunately no where near what that interface claims… but in line with the 42% presented within server manager.

Anyhoo – to sum up….. in order to see the real savings file dedup is providing, have a look in server manager, or have a look at your  “System volume informationDeDup” folder on the volume… and understand that for some volumes (depending on file type), there might not only be no savings, but it may increase your disk usage.

Dedup is something that is cool to get “for free” (i imagine most people reading this blog will have EA’s with software assurance) and will be quite useful in a number of situations…. but, as usual, the Microsoft sales machine is making it sound like the coolest thing that has ever happened (and claiming ludicrous savings – http://blogs.technet.com/b/filecab/archive/2012/05/21/introduction-to-data-deduplication-in-windows-server-2012.aspx)- so in the words of public enemy, don’t believe the hype.

August 15th – Win 2012 and Win 8 available for DL


I’ve made the (fair) assumption that server will become available at the same time as the client.

Im not convinced that many people (in the corporate space) actually care about Win 8…. part of that is because Win7 is a bloody good OS…. if Win-7-to-go existed… i think there would be even less interest in Win 8.

Server 2012 is a bit of a different matter…. while many features are a bit ho-hum, Hyper-V replica’s… there is some big cash savings to be made by swapping from VMWare and SRM… so will be interesting to see how that plays out.

Oh – and the rest of the Wave 15 products that have previews – Exchange, Lync and office 2013… well, im also quite “meh” about them too…. dont get me wrong – its not as if they are terrible – but there is just nothing particuarly exciting about any of them.

The exchange 2013 web management interface, after a few days of using it, i no longer dislike…. i fucking hate it…. moving to that after using the very good exchange 2010 management console is a huge step backwards…. i can only assume the manager that made that decision had smoked something or was busy crapping on about how great “the cloud” was to some sales idiot… or both.

With Lync, its cool to have the web client back – outside of that, while there are improvements, im a bit meh….. annoyed at the lack of authenticated sip trunk – still.

Office…. well, its office… i can imagine its hard for those guys to come up with anything new… it would be nice if they worked with the exchange team to achieve true, no impact exchange failovers… (which they well may have, but im not setting up test DAG’s and CAS arrays to find out until RTM)

On the up side, the more i use SCCM 2012 – the more it rocks. Sure, there are still improvements that could be made….. but holy shit it has come a long way since 2007, its so much more usable, responsive and quick compared to 2007. Now theres a team that got their shit together – well done.

HyperV and TCP offload

I have had issues before with Dell servers and RSS Scaling with hyper-V, so that setting now just gets disabled by default on all broadcom’s that we run into – but i started having a problem with packet loss on some other servers a few weeks ago… first up i assumed it was not with the hyper-V server, but with the clients net connection.

Now we were talking 1% packetloss over large samples…. so a 100 pings might only show 1 or 2 packets being lost… but 1000000 pings would show 1% on average.

After alot of investigation – sure enough, the problem was with the Hyper-V nic’s themselves….  on the hosts, i disabled all TCP and UDP offloading – and things got substantially better… although i still had a little bit of packet loss here and there…. disabling TCP/UDP offloading within the guest OS seemed to fix this as-well.

I’ll be turning off all network-stack offloading on all my hyperV hosts from here on in 🙂

Remote desktop gateway step by step guide

has been publsihed by MS


The most common issues with RD Gateway (that we hear of at least) seem to be around certificates, recovation checking and web SSO from XP workstations – this doc is definately seems to cover most things from having a look over it – and has a companion doc for RDWeb and websso.

Worth a look in your lab if your setting this up.

MS – some DB consistency ?

A employee of mine pointed out this quote from an SQL webpage

“HADR” is a high-availability and disaster recovery solution introduced in SQL Server Code-Named “Denali” to enable you to maximize availability for one or more of your user databases. Deploying “HADR” involves configuring one or more availability groups. Each availability group establishes the context within which a selected set of user databases can fail over as a single unit.

The availability group involves a set of failover partners, known as availability replicas. Each availability replica possesses a local copy of each of the databases in the availability group. One of these replicas, known as primary replica, maintains the primary copy of each database. The primary replica makes these databases, known as primary databases, available to users for read-write access. For each primary database, another availability replica, known as a secondary replica, maintains a failover copy of the database known as a secondary database.


along with the comment – “so SQL are going down the DAG path”….

Wouldn’t it be nice to actually see the product teams at MS talk and utilise some common technology between products.

this ofcourse could be seen as the first step to long talked about merging of the exchange ese database into using SQL as the backend.

On a slightly related note – I’ve been pretty happy with Lync 2010. Its use of replica databases on every server is a good move. The OCS product range has finally hit maturity – and while its not perfect, the product has got to the point where not only is it functionally good – but its reasonably good to admin, the client is finally pretty polished – and the all round experience is quite impressive.

Internal upgrades

Recently, i purchased a new hyper-V cheapy “server” – core i7 930, 24gb ram and 2 x sata 3 600gb raptors…. (which im running off the sata 2 controllers because of how shitty the mavell controller on the mb is… as per my previous post). Anyhoo, with this new hardware, i now have far more capacity… and have been able to move our infrastrcuture a bit closer to where it should be…

1) I implemented a Exchange 2010, 2 node DAG, with a 2 node NLB CAS/HT… which was so quick and easy to implement it was kinda scary. The only issue i ran into was… when moving a hyper-V server from one box to another, the guest’s NIC is re-created a new NIC and must be re-iped etc…. however – when moving an Exchange 2010 (any role) server between hyper-V hosts, when starting it up on the new host – the nic will not be re-detected. I pulled my hair out about this for about 30 minutes the first time it happened – then came back to the box and it had detected the NIC and all was fine. The next box i moved, i just left it, and sure enough, after 20 mins (or so) – the NIC would be detected, the driver installed, and away the server would go….. An odd issue that i didnt find a solution for, but maybe if someone else is doing the same, this post may let them know just to wait and not get stressed like i did!

2) Implemented DPM 2010, including SQL, Exchange , file and hyper-V backups. In addition i implemented a second replica server to replicate the backup sets of the first. This is the first time i’d used DPM 2010… and while there were a few minor things that could have been done better… i was fairly impressed. (but for gods sake MS, when installing the agent, just make the local backup components parts of the install)

3) The thecus N7700 NAS i got a few weeks back and also posted some initial thoughts on…. since then, ive pretty much decided the thing is a piece of junk. It becomes in-accessible daily, the web interface, while pretty, doesnt actually seem to set settings properly – and just doesnt work often. But the big issue is lack of support. The box itself has SFA logging (that i can find at least) – so finding the cause of a problem is just not possible. Next is Thecus support… who just seem to ignore support emails…. so… yes… moral of this part of the story – dont buy a thecus product…. just dodgey.

RDWeb/RDGateway – the nightmare

So i recently went into a small client to setup a single server RDWeb/RDGateway/RD Session host server… “simple enough” i thought… how wrong i was.

When having a dedicated RDWeb/RD Gateway and a farm behind it – it all fits together nicely… on a single box, i seemed to need to use SAN certs…. which for testing purposes i used the internal enterprise CA to issue and published an online responder for the revocation…. all worked fine in win7…. and i pulled my hair out for days trying to work out why the XP SSO wasnt working….

XP doesnt support online responders….  and if it cant retrieve the revocation infromation, a really fucking unhelpful error message of “the connection has been terminated because an unexpected server authentication certificate….” – FFS… there goes two wasted days of my life … i am absolutely fucking livid….

RD Gateway is great – but the logging is shithouse… it could really do with an equivalent of www.testexchangeconnectivity.com….. would be very handy (and an all in one RDC7/CredSSP/Reg entries package for XP SP3 would be nice too!)