Updated on :  15/01/2016
Relevant to:   Exchange 2010/2013/2016

---

Forest, including exchange, migrations seemed to be something I was doing every month back in the Exchange 2000/2003 days, but then they seemed to stop…. For a while.

 

Recently, in my market, there has been a bit of a spate of them again though – and searching around there are plenty of articles, but none of which I would consider comprehensive, so… this is my bash at it.

 

Tools you will need

ADMT – https://connect.microsoft.com/site1164/program8540

 

Scenario

This post will cover off the following scenarios

·         Two existing org’s merge and wish to link their mailboxes into a new resource forest, with the goal of completely migrating into the new forest over time

o   For the sake of simplicity, this document will refer to

o   DomainA.com – a company with existing AD and exchange (which has merged with DomainB.local)

o   DomainB.local – a company with existing AD and exchange (which has merged with DomainA.com)

o   Resource.com – a forest created representing the new “merged” company name

 

Assumptions

·         The person running these commands is “God” and has full access over AD and Exchange

·         Exchange 2007 or later is being utilised in all environments

·         That the person running this is proficient with AD, exchange and certificates, and I don’t need to explain basic stuff in detail

 

Versions

·         Be aware of the version requirements of your servers and your clients

·         E.g. If you have Exchange 2007 servers or Outlook 2007 clients, the highest version you can migrate to is Exchange 2013

·         For those of you coming from Exchange 2010, yes, the EAC in 2013/2016 sucks in comparison to the 2010 EMC, it just does, MS aren’t going to change it, get to use to doing everything in powershell instead

 

Creating your new forest

·         Please note that this section assumes there are no inter-company security concerns (since the companies have merged), if there are concerns, please appropriately research including required ports, trusts with selective authentication etc.

·         Create your new AD forest/domain with the DNS and NETBIOS name as appropriate for the new entity

·         Ensure there is full network connectivity between each companies DC’s

·         Create DNS conditions forwarders on each domain to the resource domain (and optionally, to the other companies domain)

o   E.g. DomainA.com.au has conditional forwarders for

§  Resource.com (the new resource domain) – 2 DNS servers for that domain

§  DomainB.local – 2 DNS servers for that domain

o   Ensure you ping resources by FQDN, from each domain

§  E.g. from domain A, ping DC1.resource.local

§  Note: you may need to restart the DNS server after adding the conditional forwarders and/or run ipconfig /flushdns

·         Create the trusts

o   Unless there are security concerns

§  create a forest trust with forest-wide authentication

§  Optionally, creating a “mesh” of trusts between all forests, this will ease sharing of resources that will inevitably happen – no matter what management tells you will happen

·         If you utilise internal certificates, ensure the other domains trust the certificate  – trusted root certificates can be distributed via group policy

·         If you utilise external certificates, ensure that the trust chain is presented correctly

 

Creating your new exchange

·         Create your new exchange organisation

·         Do all the things you would normally do for an exchange install, such as creating databases appropriately, configuring and testing external access to ActiveSync/owa etc.

 

Preparing AD and exchange – pre-migration tasks

·         Create a new server in the resource.com for ADMT, as of 8/1/2016, I would suggest a 2012 R2 standard server

·         Install ADMT on this server

·         (optional – if you will be migrating completely into this domain eventually)

o   Install the Password Export server on one DC in each DomainA.com and DomainB.local – this will require a reboot after install

o   Run the following command on the ADMT server to generate a PES key

o   C:\Windows\Admt\admt.exe key /option:create /sourcedomain:<SourceDomain> /keyfile:<KeyFilePath>

·         Enable SIDHistory to traverse your trust

o   Netdom trust TrustingDomainName /domain:TrustedDomainName /quarantine:No /usero:domainadministratorAcct /passwordo:domainadminpwd

o   See this article for full details – https://blogs.technet.microsoft.com/csstwplatform/2010/05/06/how-to-disabling-sid-filter-quarantining-allowing-sid-history/

·         Enable MRSProxy on exchange

o   This only needs to be done on the “source” exchange server…. So in our scenario, DomainA and DomainB would need it enabled as migrations would be initiated from resource.com, however

§  I prefer running tests prior to migration, moving mailboxes into and back out of the resource domain

§  Due to version requirements, move requests will need to be initiated from the highest version exchange server

§  For this reason, I enable MRSProxy on one exchange server in each forest

§  To enable the MRSProxy run “Get-WebServicesVirtualDirectory | Set-WebServicesVirtualDirectory -Identity “EWS (Default Web Site)” -MRSProxyEnabled $true -MRSProxyMaxConnections 50”

o   Ensure you can get to the MRSProxy without a certificate warning, if you get a certificate warning, your migrations will fail

§  Open IE and point to https://<ews internal FQDN>/EWS/mrsproxy.svc

·         If you get a certificate warning verify

o   If the certificate was issued from an internal CA, verify that the foreign domains trust that CA issuing the certificates

o   That you are entering the correct EWS name, this might not necessarily be the same as the server name

§  Run “Get-WebServicesVirtualDirectory | select InternalURL”, this is the name EWS is presenting as

§  This is the error you will get if you try and migrate and the certificate is not trusted or the name does not match up

·         The call to ‘https://<EWSname>/EWS/mrsproxy.svc’ failed. Error details: Could not establish trust relationship for the SSL/TLS secure channel with authority ‘<EWSName>. –> The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. –> The remote certificate is invalid according to the validation procedure

·         Be aware that migration will create a truckload of logs…. So, you have a few options here

o   Ensure there is truckloads of space available on the logs drive

§  Remember, you can use compression…. Exchange logs compress really well – but this has a CPU hit… again, if you use this, turn it off immediately after the migration

o   Temporarily enable circular logging – only during migration

§  Note: Yes, circular logging is evil…. However, if you simply don’t have the disk space, the fact is it’s a viable option <but if you f*ck up your migration and can’t recover, don’t come crying to me>

o   “How much is truckloads ?”

§  Unfortunately this depends on the size of your environment, so there is no great guide

§  As an exchange admin, you’re probably aware of your database size. As a rule of thumb, allow for 2 x your database size

 

Migration speed

·         Understand that exchange mailbox moves go at around about the same pace as a snail, with weight on his back, obstacles in his way…. To use a technical term, they are f*cking slow – https://technet.microsoft.com/en-us/library/dn592150(v=exchg.150).aspx

·         There are some things you can do to speed them up (from f*cking unbearably slow to f*cking slightly-less unbearably slow)

o   Ensure there is no file level antivirus running on the migrating servers

o   Ensure there are not backup jobs running on the migrating servers

o   Ensure the servers are appropriately resourced…. i.e. have a look at performance monitor during a small test migration. Is there lots of hard faults ? increase memory. Is CPU for store.exe hitting 100%, increase CPU etc.

o   Disable indexing

§  Set-MailboxDatabase “<database name>” -IndexEnabled:$False

§  Note: This should be re-enabled once the migration has completed

o   Disable resource throttling for EWS

§  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\MSExchange ResourceHealth\EWS (DWORD) – Change the Value from “1” to “0”

§  Restart the “Microsoft exchange mailbox replication” service

§  Note: This should be re-enabled once the migration has completed

o   Change the concurrency settings

§  <Drive where exchange is installed>:\Program Files\Microsoft\Exchange Server\V15\Bin\ MSExchangeMailboxReplication.exe.config

·         Change the following entries

·         MaxActiveMovesPerSourceMDB=”20″

·         MaxActiveMovesPerTargetMDB=”20″

·         MaxActiveMovesPerSourceServer=”100″

·         MaxActiveMovesPerTargetServer=”100″

·         MaxTotalRequestsPerMRS=”100″

·         ExportBufferSizeKB=”10240″

·         Restart the “Microsoft exchange mailbox replication” service

 

Documentation

·         Before starting your migration, I recommend documenting certain settings, this will both help you to migrate, and help you to update things should something accidentally get missed

·         Get-Contact | select Name,WindowsEmailaddress >>C:\Temp\Contacts.txt

·         Get-Mailbox | Select Alias,ForwardingAddress | where {$_.ForwardingAddress –ne $Null} | out-string –width 150 >>C:\Temp\Forwarders.txt

·         Get-Mailbox | Select Alias,GrantSendOnBehalfTo | where {$_. GrantSendOnBehalfTo –ne $Null} |out-string –width 150 >>C:\Temp\GrantSendOnBehalfTo.txt

·         Get-Mailbox | Select Alias,*quota* | ft | out-string –width 150 >>C:\Temp\Quotas.txt

·         Get-CASMailbox >>C:\Temp\UserCAS.txt

·         Get-Mailbox | Get-MailboxPermission | where {$_.user.tostring() -ne “NT AUTHORITY\SELF” -and $_.IsInherited -eq $false} | Select Identity,User,@{Name=’Access Rights’;Expression={[string]::join(‘, ‘, $_.AccessRights)}} | Export-Csv -NoTypeInformation C:\temp\permissions.csv

·         Get-mailbox | where {$.HiddenFromAddressListsEnabled –eq $true} >> C:\temp\Hidden.txt

·         Get-Mailbox -ResultSize Unlimited | Where {$_.EmailAddressPolicyEnabled -eq $False} | select DisplayName,PrimarySMTPAddress

·         The excellent script available from here – https://gallery.technet.microsoft.com/office/Exchange-Proxy-Address-98d05872 to get all additional addresses on each account (this script required powershell 3 or above – so you may need to install this on Windows 2008 R2/Exchange 2007/2010 servers first)

·         Get-transportrule | fl

·         Get-RetentionPolicy | fl

·         Get-recieveconnector | select name,RemoteIPRanges,permissiongroup,authmechanism | fl

·         For small amounts of public folders – Get-PublicFolder \ -GetChildren | Get-PublicFolderClientPermission

o   Public folders seem to be pretty rare these days, so I haven’t had to work out a method for large numbers of PF’s

 

I’m not claiming this is a complete list, but it has served me well on my migrations

 

Migration

Step 1 –  prepping exchange

·         Add all accepted domains that will be accepted into the new exchange

·         Prepare databases and associated limits in the new exchange (use quotas.txt from the above step to help if necessary)

·         Create email address policies

o   To try auto-stamp all required addresses via policy – it is not good practice to disable email address policies on mailboxes

·         Create send connectors

·         Create receive connects, using information gathered from Get-recieveconnector | select name,RemoteIPRanges,permissiongroup,authmechanism | fl

·         Create an transport rules that are required in the new exchange

·         Esnure all web services URLs are populated (internal/external)

·         If authentication is being changed, you need to change authentication for OAB virtual directory as well, but has to be done via powershell (all others can be done via GUI)

o   If exchange 2016, you also need to change the auth on the MAPI virtual directory (via powershell – to match Outlook Anywhere)

·         Run Test-OutlookWebServices/Test-OutlookConnectivity

·         Use testexchangeconnectivity.com to verify services to the outside world

 

Step 2 – migrating test mailboxes

·         Create some test mailboxes to migrate (or you may already have some) on one (or both) of the source forests

·         On the target forest, create a Test OU for target objects

·         On a test workstation, logon and see that auto-discover connects you to your mailbox

·         On the target exchange server, open an elevated exchange command shell

o   Cd $exscripts

o   $Rcred = Get-Credential

§  Enter credentials for the remote exchange server

o   First, we need to create the account in the resource forest that the mailbox will be associated with

§  For a “normal” mailbox

·         .\Prepare-MoveRequest.ps1 -Identity “alias” -RemoteForestDomainController “sourceDC” -RemoteForestCredential $Rcred -TargetMailUserOU “OUDN”

§  For a linked mailbox

·         .\Prepare-MoveRequest.ps1 -Identity “alias” -RemoteForestDomainController “sourceDC” -RemoteForestCredential $Rcred -LinkedMailUser -TargetMailUserOU “OUDN”

o   Then we actually move the mailbox

§  New-MoveRequest -Identity “alias” -Remote -TargetDatabase “DBName″ -RemoteGlobalCatalog “sourcedomainDC” -RemoteCredential $Rcred -TargetDeliveryDomain “DomainSuffix” -RemoteHostName “SourceexchangeEWSName”

o   Now we use ADMT to migrate the account (or migrate SID history) – depending on which scenario you are undertaking

§  User migration wizard

§  Select source and destination domain

§  Select users from domain

·         Select your test user(s)

§  Select a target OU

§  If you are performing a complete forest migration, migrating passwords is almost certainly a requirement

§  If you are only migrating mailboxes (linked scenario), then migrating passwords is not required

§  Ensure “Migrate user’s SID to target domain” is selected

§  Enter credentials for the source domain

§  Select “Migrate associated user groups”, “update previously migrated objects” and “fix users’ group membership”

§  Select “Migrate and merge existing objects” – This step is vital – as we need to merge the SIDhistory attribute onto the account we created with Prepare-MoveRequest.ps1

§  Our account now has SIDhistory, which you can verify via ADSIEdit (or any other method, such as ldp etc.) if you wish

·         Go back to your test workstation, you will now likely have a message “Your exchange administrator has made a change that requires you to restart outlook”

o   The next part depends on your domains and autodiscover

o   If you are migrating into a completely new domain, you can use a combination of the following methods

§  Create an autodiscover.resource.com DNS A record or

§  Utilise a hosts file locally if you cannot create the domain in DNS for various reasons

§  Use GPO templates at https://support.microsoft.com/en-au/kb/2612922 , please note that only templates for 2007 and 2010 are available, these settings are built in for 2013 and 2016, so get them from the admx downloads for those products

o   Once you have autodsicover sorted, the mailbox should automatically discover it’s new server and the outlook profile should migrate with a simple restart

 

Step 3 – mailbox migration

·         Get a list of all the mailbox alias’ from each org

o   Get-mailbox | select alias >>C:\temp\org1.txt (and org2.txt)

·         On the target exchange server, open an elevated exchange command shell

o   Cd $exscripts

o   $Rcred = Get-Credential

o   Create the objects to migrate to

§  $Users = get-content “C:\temp\org1.txt”

§  For “normal mailboxes”

·         Foreach ($User in $Users) {.\Prepare-MoveRequest.ps1 –identity $User -RemoteForestDomainController “SourceDC” -RemoteForestCredential $Rcred -TargetMailUserOU “DN of new user OU”}

§  For linked mailboxes

·         Foreach ($User in $Users) {.\Prepare-MoveRequest.ps1 –identity $User -RemoteForestDomainController “SourceDC” -RemoteForestCredential $Rcred -LinkedMailUser -TargetMailUserOU “DN of new user OU”}

o   Migrate the mailboxes

§  Get-content “C:\temp\org1.txt” | New-MoveRequest -Remote -TargetDatabase “DBName” -RemoteGlobalCatalog “SourceDC” -RemoteCredential $Rcred -TargetDeliveryDomain “DomainSuffix” -RemoteHostName “SourceexchangeEWSName” – suspendwhenreadytocomplete

o   Wait…. And wait and wait….

o   Check the status

§  Get-moverequest | get-moverequeststatistics

§  Get-moverequest –movestatus “inprogress” | get-moverequeststatistics

 

Step 4 – handling errors

§  In any migration, you will get errors

o   If the Prepare-MoveRequest.ps1 script fails, which it will for some users, I have found

§  If the shell is not run as administrator, it will not update the legacyexchangeDN, and therefore the mailbox move will fail stating that a required attribute does not exist

§  If the account associated with the mailbox is disabled, the error “Source Mailbox is invalid because it is disabled but did not set msExchMasterAccountSid”

·         Enable the account, then re-run. The account can be disabled again after being re-run

o   If the new-moverequest fails

§  If the mailbox has email addresses that don’t match the –TargetDeliveryDomain suffix, they will not be added, simply re-run the command with an updated targetdeliverydomain for that mailbox, or list of mailboxes

o   Some mailboxes will start migrating, but fail during migration, this is generally due to

§  Exceeded number of baditems

§  Exceeded number of large messages

§  It is possible to see these by “get-moveRequest –movestatus “failed”

§  More detail can be obtained by using get-moveRequest –movestatus “failed” | get-moverequeststatistics –includereport | fl >>C:\Temp\Failreport.txt

·         You can then see why the mailbox has failed

§  From there you have a couple of options

·         Migrate skipping corrupt items and items over 25Mb

·         Run a mailbox health check on the source mailbox to try and fix corrupted items (this rarely works in my experience)

·         Search for large items (i.e. items over 50mb) – and store them somewhere more appropriate

§  If you choose to ignore baditems and items of 50mb, you can run

·         get-moveRequest –movestatus “failed” | set-moverequest -AcceptLargeDataLoss –baditemlimit 99999 –largeitemlimit 99999

·         get-moveRequest –movestatus “failed” | resume-moverequest

·         Note: This will result in data loss – be sure you want to do this before you do it!

§  If you choose to find large items in the source mailbox and store them more appropriately

·         Get-Mailbox -ResultSize Unlimited | Get-MailboxFolderStatistics -IncludeAnalysis -FolderScope All | Where-Object {$_.TopSubjectSize -gt 25MB} | Select-Object Identity, TopSubject, TopSubjectSize | Export-CSV -path “C:\temp\report.csv” -notype

·         Or https://gallery.technet.microsoft.com/PowerShell-Script-Office-54d367ea

§  If you choose to find corrupted items

·         New-MailboxRepairRequest -Mailbox “alias” -CorruptionType ProvisionedFolder, SearchFolder, AggregateCounts Folderview\

o   Some mailbox moves may appear to be stuck

§  When running get-moverequest –movestatus “inprogress” | get-moverequeststatistics, the status detail shows as “relinquishedwlmStall”

§  Running get-moverequest –identity <one of the stalled mailbox alias’> | get-moverequeststatistics –includereport | fl

·         At the bottom the following test is shown “Relinquishing job because of large delays due to unfavourable server health or budget limitations”

§  Check the database content index state, it is most likely “failed and suspended”, this means you didn’t disable indexing on this database before migration.

·         stop-service MSExchangeFastSearch

·         stop-service HostControllerService

·         Get-MailboxDatabase <DBName> | select EdbFilePath

·         Navigate to the directory

·         Delete the guid path under the same dir

·         Set-MailboxDatabase “<database name>” -IndexEnabled:$False

·         start-service MSExchangeFastSearch

·         start-service HostControllerService

 

Step 5 – Finishing off mailbox migration

§  Once you are at the point where every move request is in the state “autosuspended”, you are ready to complete

§  Suspend mail flow into your organisation, so waiting messages will get queued

§  Get-moverequest | set-moverequest –suspendwhenreadytocomplete:$false

§  Get-Moverequest | resume-moverequest

 

 

Step 6 – ADMT

·         We can now use ADMT to migrate SIDHistory and group memberships into the target domain

·         Start ADMT

·         User account migration wizard

o   In step 3 we grabbed a list of all mailbox alias  – Get-mailbox | select alias >>C:\temp\org1.txt

o   We are going to use this list as the list of accounts we need to migrate (as we only want to migrate users with mailboxes)

o   There is an issue with this method in that sometimes the mailbox alias will not match the username

§  This generally occurs when a user has changed their name (such as when someone gets married) and the admin was a little bit lazy and didn’t update it everywhere

§  I’ve also found some instances where admins change the mailbox name of shared mailboxes as the business requests, but doesn’t change other items, such as the alias, username etc

§  ADMT will help you identify these accounts in a later step

·         Select your source and destination domain

o   If you are suing password export, the DC selected must be the DC with the PES (pass word export server) installed

·         Read objects from an include file

o   Select :\temp\org1.txt (or org2 as appropriate)

·         At this point you will get an error popping up if any of the accounts specified cannot be found in the source domain…. At the point you have two options

o   Go back and fix the source account info – this is what I recommend – however, if you are changing usernames, you obviously may need to notify users (although if its for a resource mailbox that no-one directly logs into, you will not)

o   Modify the text file to match the names of these accounts

·         Select a target OU

·         Select if you wish to migrate passwords or generate complex passwords

·         Select the target account state (if this is resource forest, the user accounts should be disabled in the destination domain)

·         Most importantly – ensure “Migrate users SID to target domain”

·         Enter the username and password

·         Away you go

 

Step 7 – migrating DL’s

·         You will already have DL’s created and populated, thanks to the ADMT in step 6, however these groups will be lacking mail enabled-ness and SMTP addresses

·         This article – http://blogs.technet.com/b/sachinf/archive/2014/03/17/how-to-migrate-distribution-groups-across-a-forest.aspx – contains some details on using some Microsoft provided scripts for migrating DL’s… but no great surprise being MS…. It does the job, but, well, it doesn’t make it easy

·         Download the toolkit from http://blogs.technet.com/b/exchange/archive/2012/02/03/released-migrating-from-exchange-server-2010-in-hosting-mode-to-exchange-server-2010-sp2-whitepaper.aspx

·         Extract the lot, and then go to the directory \HostingMode_to_SP2_SampleMigrationScripts

·         Copy this directory to your source and target domains

o   .\QueryCustomerOrg.ps1 <domain name>

o   Edit the following

§  $PreferredDC = <DC FQDNS>

§  $DomainID = <NetBIOS name>

§  $DomainSuffix = <domain suffix>

§  $HostingOU = <OU>

§  $BaseSearchPath = <DNS of top level>

§  $OutputFile = “C:\Scripts\ ”                                         <if you don’t plan on running from C:\scripts, change this>

§  $xml = (get-content “C:\Scripts\”)                 <if you don’t plan on running from C:\scripts, change this>

§  $searcher.SearchScope = “Subtree”                        Change to Subtree if you want to the script to be of any use

o   The script must be run with an argument of the OU to check, so simply enter your OU (this is why you put the top level in the $BaseSerachPath)

o   Run the script

§  Then run CreateADMTGroupList.ps1 <source to xml generate above> <NetBIOS domain name>

§  Copy the xml and group list text to the destination domain

o   On the destination server

§  Edit Restoregroupattributes.ps1 with your DC name

§  Run the script .\ResotreGroupAttributes.ps1 <path to xml file>

o   Rinse and repeat for all other domains you are migrating

·         In my move from 2010/2013 to a 2013 resource forest, I found a few issues with this script

o   Not all groups got converted to mail-enabled universal groups

o   Other properties of the groups, such as delivery management were not copied

 

Step 8 – Fixing Issues

I had the following issues in my migrations:

·         Auto-mapping of mailboxes where the user has full permission no longer auto-mapped

o   This was a factor of two things

§  The attribute MSexchDelegateLinkList was not migrated on the shared mailboxes. It was a relatively short list for my migrations, so I did this manually

§  One of the “old” domains, which was no longer in use, did not have autodsicover pointed to the new box. This seemed to break auto-mapping where the shared mailbox was an “old” mailbox which was only used for reference purposes

·         The client used airwatch to manage mobile devices

o   This updated settings, but required users to re-enter their password

o   As you can imagine, for some users, this was fine…. For others, the sky had fallen/it was the end of the world

o   I left this to the internal team to manage, and I probably should have been more involved and understood the impacts

·         One account had a Sid in the “linked account” instead of the account (not sure why), needed to be re-updated using

o   Get-mailbox -identity <name> | set-mailbox linkedmatseraccount <linked account name> -linkedodmaincontroller <dc name in source domain>

 

Appendix – things I would do before my next Exchange inter-org migration (if one ever comes up again)

·         Users can “favourite” shared contacts from public folders, this does not stick once those PF’s are migrated to a new exchange org.

o   Write a script to discover this setting, and re-instate it after migration

·         Write a script to export “MSExchDelegateLinkList” and import it within the new forest to address mailbox auto-mapping

·         Write a script to migrate contacts

·         Write a better script to migrate DL’s from forest to forest, the one in step 7 is intended for 2010, so misses some 2013 attributes – and is quite clunky