Exchange inter-forest migration tips

Updated on :  15/01/2016
Relevant to:   Exchange 2010/2013/2016


Forest, including exchange, migrations seemed to be something I was doing every month back in the Exchange 2000/2003 days, but then they seemed to stop…. For a while.

 

Recently, in my market, there has been a bit of a spate of them again though – and searching around there are plenty of articles, but none of which I would consider comprehensive, so… this is my bash at it.

 

Tools you will need

ADMT – https://connect.microsoft.com/site1164/program8540

 

Scenario

This post will cover off the following scenarios

·         Two existing org’s merge and wish to link their mailboxes into a new resource forest, with the goal of completely migrating into the new forest over time

o   For the sake of simplicity, this document will refer to

o   DomainA.com – a company with existing AD and exchange (which has merged with DomainB.local)

o   DomainB.local – a company with existing AD and exchange (which has merged with DomainA.com)

o   Resource.com – a forest created representing the new “merged” company name

 

Assumptions

·         The person running these commands is “God” and has full access over AD and Exchange

·         Exchange 2007 or later is being utilised in all environments

·         That the person running this is proficient with AD, exchange and certificates, and I don’t need to explain basic stuff in detail

 

Versions

·         Be aware of the version requirements of your servers and your clients

·         E.g. If you have Exchange 2007 servers or Outlook 2007 clients, the highest version you can migrate to is Exchange 2013

·         For those of you coming from Exchange 2010, yes, the EAC in 2013/2016 sucks in comparison to the 2010 EMC, it just does, MS aren’t going to change it, get to use to doing everything in powershell instead

 

Creating your new forest

·         Please note that this section assumes there are no inter-company security concerns (since the companies have merged), if there are concerns, please appropriately research including required ports, trusts with selective authentication etc.

·         Create your new AD forest/domain with the DNS and NETBIOS name as appropriate for the new entity

·         Ensure there is full network connectivity between each companies DC’s

·         Create DNS conditions forwarders on each domain to the resource domain (and optionally, to the other companies domain)

o   E.g. DomainA.com.au has conditional forwarders for

§  Resource.com (the new resource domain) – 2 DNS servers for that domain

§  DomainB.local – 2 DNS servers for that domain

o   Ensure you ping resources by FQDN, from each domain

§  E.g. from domain A, ping DC1.resource.local

§  Note: you may need to restart the DNS server after adding the conditional forwarders and/or run ipconfig /flushdns

ConditionalFowarders

·         Create the trusts

o   Unless there are security concerns

§  create a forest trust with forest-wide authentication

§  Optionally, creating a “mesh” of trusts between all forests, this will ease sharing of resources that will inevitably happen – no matter what management tells you will happen

·         If you utilise internal certificates, ensure the other domains trust the certificate  – trusted root certificates can be distributed via group policy

·         If you utilise external certificates, ensure that the trust chain is presented correctly

 

Creating your new exchange

·         Create your new exchange organisation

·         Do all the things you would normally do for an exchange install, such as creating databases appropriately, configuring and testing external access to ActiveSync/owa etc.

 

Preparing AD and exchange – pre-migration tasks

·         Create a new server in the resource.com for ADMT, as of 8/1/2016, I would suggest a 2012 R2 standard server

·         Install ADMT on this server

·         (optional – if you will be migrating completely into this domain eventually)

o   Install the Password Export server on one DC in each DomainA.com and DomainB.local – this will require a reboot after install

o   Run the following command on the ADMT server to generate a PES key

o   C:\Windows\Admt\admt.exe key /option:create /sourcedomain:<SourceDomain> /keyfile:<KeyFilePath>

·         Enable SIDHistory to traverse your trust

o   Netdom trust TrustingDomainName /domain:TrustedDomainName /quarantine:No /usero:domainadministratorAcct /passwordo:domainadminpwd

o   See this article for full details – https://blogs.technet.microsoft.com/csstwplatform/2010/05/06/how-to-disabling-sid-filter-quarantining-allowing-sid-history/

·         Enable MRSProxy on exchange

o   This only needs to be done on the “source” exchange server…. So in our scenario, DomainA and DomainB would need it enabled as migrations would be initiated from resource.com, however

§  I prefer running tests prior to migration, moving mailboxes into and back out of the resource domain

§  Due to version requirements, move requests will need to be initiated from the highest version exchange server

§  For this reason, I enable MRSProxy on one exchange server in each forest

§  To enable the MRSProxy run “Get-WebServicesVirtualDirectory | Set-WebServicesVirtualDirectory -Identity “EWS (Default Web Site)” -MRSProxyEnabled $true -MRSProxyMaxConnections 50”

o   Ensure you can get to the MRSProxy without a certificate warning, if you get a certificate warning, your migrations will fail

§  Open IE and point to https://<ews internal FQDN>/EWS/mrsproxy.svc

·         If you get a certificate warning verify

o   If the certificate was issued from an internal CA, verify that the foreign domains trust that CA issuing the certificates

o   That you are entering the correct EWS name, this might not necessarily be the same as the server name

§  Run “Get-WebServicesVirtualDirectory | select InternalURL”, this is the name EWS is presenting as

§  This is the error you will get if you try and migrate and the certificate is not trusted or the name does not match up

·         The call to ‘https://<EWSname>/EWS/mrsproxy.svc’ failed. Error details: Could not establish trust relationship for the SSL/TLS secure channel with authority ‘<EWSName>. –> The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. –> The remote certificate is invalid according to the validation procedure

·         Be aware that migration will create a truckload of logs…. So, you have a few options here

o   Ensure there is truckloads of space available on the logs drive

§  Remember, you can use compression…. Exchange logs compress really well – but this has a CPU hit… again, if you use this, turn it off immediately after the migration

o   Temporarily enable circular logging – only during migration

§  Note: Yes, circular logging is evil…. However, if you simply don’t have the disk space, the fact is it’s a viable option <but if you f*ck up your migration and can’t recover, don’t come crying to me>

o   “How much is truckloads ?”

§  Unfortunately this depends on the size of your environment, so there is no great guide

§  As an exchange admin, you’re probably aware of your database size. As a rule of thumb, allow for 2 x your database size

 

Migration speed

·         Understand that exchange mailbox moves go at around about the same pace as a snail, with weight on his back, obstacles in his way…. To use a technical term, they are f*cking slow – https://technet.microsoft.com/en-us/library/dn592150(v=exchg.150).aspx

·         There are some things you can do to speed them up (from f*cking unbearably slow to f*cking slightly-less unbearably slow)

o   Ensure there is no file level antivirus running on the migrating servers

o   Ensure there are not backup jobs running on the migrating servers

o   Ensure the servers are appropriately resourced…. i.e. have a look at performance monitor during a small test migration. Is there lots of hard faults ? increase memory. Is CPU for store.exe hitting 100%, increase CPU etc.

o   Disable indexing

§  Set-MailboxDatabase “<database name>” -IndexEnabled:$False

§  Note: This should be re-enabled once the migration has completed

o   Disable resource throttling for EWS

§  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\MSExchange ResourceHealth\EWS (DWORD) – Change the Value from “1” to “0”

§  Restart the “Microsoft exchange mailbox replication” service

§  Note: This should be re-enabled once the migration has completed

o   Change the concurrency settings

§  <Drive where exchange is installed>:\Program Files\Microsoft\Exchange Server\V15\Bin\ MSExchangeMailboxReplication.exe.config

·         Change the following entries

·         MaxActiveMovesPerSourceMDB=”20″

·         MaxActiveMovesPerTargetMDB=”20″

·         MaxActiveMovesPerSourceServer=”100″

·         MaxActiveMovesPerTargetServer=”100″

·         MaxTotalRequestsPerMRS=”100″

·         ExportBufferSizeKB=”10240″

·         Restart the “Microsoft exchange mailbox replication” service

 

Documentation

·         Before starting your migration, I recommend documenting certain settings, this will both help you to migrate, and help you to update things should something accidentally get missed

·         Get-Contact | select Name,WindowsEmailaddress >>C:\Temp\Contacts.txt

·         Get-Mailbox | Select Alias,ForwardingAddress | where {$_.ForwardingAddress –ne $Null} | out-string –width 150 >>C:\Temp\Forwarders.txt

·         Get-Mailbox | Select Alias,GrantSendOnBehalfTo | where {$_. GrantSendOnBehalfTo –ne $Null} |out-string –width 150 >>C:\Temp\GrantSendOnBehalfTo.txt

·         Get-Mailbox | Select Alias,*quota* | ft | out-string –width 150 >>C:\Temp\Quotas.txt

·         Get-CASMailbox >>C:\Temp\UserCAS.txt

·         Get-Mailbox | Get-MailboxPermission | where {$_.user.tostring() -ne “NT AUTHORITY\SELF” -and $_.IsInherited -eq $false} | Select Identity,User,@{Name=’Access Rights’;Expression={[string]::join(‘, ‘, $_.AccessRights)}} | Export-Csv -NoTypeInformation C:\temp\permissions.csv

·         Get-mailbox | where {$.HiddenFromAddressListsEnabled –eq $true} >> C:\temp\Hidden.txt

·         Get-Mailbox -ResultSize Unlimited | Where {$_.EmailAddressPolicyEnabled -eq $False} | select DisplayName,PrimarySMTPAddress

·         The excellent script available from here – https://gallery.technet.microsoft.com/office/Exchange-Proxy-Address-98d05872 to get all additional addresses on each account (this script required powershell 3 or above – so you may need to install this on Windows 2008 R2/Exchange 2007/2010 servers first)

·         Get-transportrule | fl

·         Get-RetentionPolicy | fl

·         Get-recieveconnector | select name,RemoteIPRanges,permissiongroup,authmechanism | fl

·         For small amounts of public folders – Get-PublicFolder \ -GetChildren | Get-PublicFolderClientPermission

o   Public folders seem to be pretty rare these days, so I haven’t had to work out a method for large numbers of PF’s

 

I’m not claiming this is a complete list, but it has served me well on my migrations

 

Migration

Step 1 –  prepping exchange

·         Add all accepted domains that will be accepted into the new exchange

·         Prepare databases and associated limits in the new exchange (use quotas.txt from the above step to help if necessary)

·         Create email address policies

o   To try auto-stamp all required addresses via policy – it is not good practice to disable email address policies on mailboxes

·         Create send connectors

·         Create receive connects, using information gathered from Get-recieveconnector | select name,RemoteIPRanges,permissiongroup,authmechanism | fl

·         Create an transport rules that are required in the new exchange

·         Esnure all web services URLs are populated (internal/external)

·         If authentication is being changed, you need to change authentication for OAB virtual directory as well, but has to be done via powershell (all others can be done via GUI)

o   If exchange 2016, you also need to change the auth on the MAPI virtual directory (via powershell – to match Outlook Anywhere)

·         Run Test-OutlookWebServices/Test-OutlookConnectivity

·         Use testexchangeconnectivity.com to verify services to the outside world

 

Step 2 – migrating test mailboxes

·         Create some test mailboxes to migrate (or you may already have some) on one (or both) of the source forests

·         On the target forest, create a Test OU for target objects

·         On a test workstation, logon and see that auto-discover connects you to your mailbox

·         On the target exchange server, open an elevated exchange command shell

o   Cd $exscripts

o   $Rcred = Get-Credential

§  Enter credentials for the remote exchange server

o   First, we need to create the account in the resource forest that the mailbox will be associated with

§  For a “normal” mailbox

·         .\Prepare-MoveRequest.ps1 -Identity “alias” -RemoteForestDomainController “sourceDC” -RemoteForestCredential $Rcred -TargetMailUserOU “OUDN”

§  For a linked mailbox

·         .\Prepare-MoveRequest.ps1 -Identity “alias” -RemoteForestDomainController “sourceDC” -RemoteForestCredential $Rcred -LinkedMailUser -TargetMailUserOU “OUDN”

o   Then we actually move the mailbox

§  New-MoveRequest -Identity “alias” -Remote -TargetDatabase “DBName″ -RemoteGlobalCatalog “sourcedomainDC” -RemoteCredential $Rcred -TargetDeliveryDomain “DomainSuffix” -RemoteHostName “SourceexchangeEWSName”

o   Now we use ADMT to migrate the account (or migrate SID history) – depending on which scenario you are undertaking

§  User migration wizard

§  Select source and destination domain

§  Select users from domain

·         Select your test user(s)

§  Select a target OU

§  If you are performing a complete forest migration, migrating passwords is almost certainly a requirement

§  If you are only migrating mailboxes (linked scenario), then migrating passwords is not required

§  Ensure “Migrate user’s SID to target domain” is selected

§  Enter credentials for the source domain

§  Select “Migrate associated user groups”, “update previously migrated objects” and “fix users’ group membership”

§  Select “Migrate and merge existing objects” – This step is vital – as we need to merge the SIDhistory attribute onto the account we created with Prepare-MoveRequest.ps1

§  Our account now has SIDhistory, which you can verify via ADSIEdit (or any other method, such as ldp etc.) if you wish

·         Go back to your test workstation, you will now likely have a message “Your exchange administrator has made a change that requires you to restart outlook”

o   The next part depends on your domains and autodiscover

o   If you are migrating into a completely new domain, you can use a combination of the following methods

§  Create an autodiscover.resource.com DNS A record or

§  Utilise a hosts file locally if you cannot create the domain in DNS for various reasons

§  Use GPO templates at https://support.microsoft.com/en-au/kb/2612922 , please note that only templates for 2007 and 2010 are available, these settings are built in for 2013 and 2016, so get them from the admx downloads for those products

o   Once you have autodsicover sorted, the mailbox should automatically discover it’s new server and the outlook profile should migrate with a simple restart

 

Step 3 – mailbox migration

·         Get a list of all the mailbox alias’ from each org

o   Get-mailbox | select alias >>C:\temp\org1.txt (and org2.txt)

·         On the target exchange server, open an elevated exchange command shell

o   Cd $exscripts

o   $Rcred = Get-Credential

o   Create the objects to migrate to

§  $Users = get-content “C:\temp\org1.txt”

§  For “normal mailboxes”

·         Foreach ($User in $Users) {.\Prepare-MoveRequest.ps1 –identity $User -RemoteForestDomainController “SourceDC” -RemoteForestCredential $Rcred -TargetMailUserOU “DN of new user OU”}

§  For linked mailboxes

·         Foreach ($User in $Users) {.\Prepare-MoveRequest.ps1 –identity $User -RemoteForestDomainController “SourceDC” -RemoteForestCredential $Rcred -LinkedMailUser -TargetMailUserOU “DN of new user OU”}

o   Migrate the mailboxes

§  Get-content “C:\temp\org1.txt” | New-MoveRequest -Remote -TargetDatabase “DBName” -RemoteGlobalCatalog “SourceDC” -RemoteCredential $Rcred -TargetDeliveryDomain “DomainSuffix” -RemoteHostName “SourceexchangeEWSName” – suspendwhenreadytocomplete

o   Wait…. And wait and wait….

o   Check the status

§  Get-moverequest | get-moverequeststatistics

§  Get-moverequest –movestatus “inprogress” | get-moverequeststatistics

 

Step 4 – handling errors

§  In any migration, you will get errors

o   If the Prepare-MoveRequest.ps1 script fails, which it will for some users, I have found

§  If the shell is not run as administrator, it will not update the legacyexchangeDN, and therefore the mailbox move will fail stating that a required attribute does not exist

§  If the account associated with the mailbox is disabled, the error “Source Mailbox is invalid because it is disabled but did not set msExchMasterAccountSid”

·         Enable the account, then re-run. The account can be disabled again after being re-run

o   If the new-moverequest fails

§  If the mailbox has email addresses that don’t match the –TargetDeliveryDomain suffix, they will not be added, simply re-run the command with an updated targetdeliverydomain for that mailbox, or list of mailboxes

o   Some mailboxes will start migrating, but fail during migration, this is generally due to

§  Exceeded number of baditems

§  Exceeded number of large messages

§  It is possible to see these by “get-moveRequest –movestatus “failed”

§  More detail can be obtained by using get-moveRequest –movestatus “failed” | get-moverequeststatistics –includereport | fl >>C:\Temp\Failreport.txt

·         You can then see why the mailbox has failed

§  From there you have a couple of options

·         Migrate skipping corrupt items and items over 25Mb

·         Run a mailbox health check on the source mailbox to try and fix corrupted items (this rarely works in my experience)

·         Search for large items (i.e. items over 50mb) – and store them somewhere more appropriate

§  If you choose to ignore baditems and items of 50mb, you can run

·         get-moveRequest –movestatus “failed” | set-moverequest -AcceptLargeDataLoss –baditemlimit 99999 –largeitemlimit 99999

·         get-moveRequest –movestatus “failed” | resume-moverequest

·         Note: This will result in data loss – be sure you want to do this before you do it!

§  If you choose to find large items in the source mailbox and store them more appropriately

·         Get-Mailbox -ResultSize Unlimited | Get-MailboxFolderStatistics -IncludeAnalysis -FolderScope All | Where-Object {$_.TopSubjectSize -gt 25MB} | Select-Object Identity, TopSubject, TopSubjectSize | Export-CSV -path “C:\temp\report.csv” -notype

·         Or https://gallery.technet.microsoft.com/PowerShell-Script-Office-54d367ea

§  If you choose to find corrupted items

·         New-MailboxRepairRequest -Mailbox “alias” -CorruptionType ProvisionedFolder, SearchFolder, AggregateCounts Folderview\

o   Some mailbox moves may appear to be stuck

§  When running get-moverequest –movestatus “inprogress” | get-moverequeststatistics, the status detail shows as “relinquishedwlmStall”

§  Running get-moverequest –identity <one of the stalled mailbox alias’> | get-moverequeststatistics –includereport | fl

·         At the bottom the following test is shown “Relinquishing job because of large delays due to unfavourable server health or budget limitations”

§  Check the database content index state, it is most likely “failed and suspended”, this means you didn’t disable indexing on this database before migration.

·         stop-service MSExchangeFastSearch

·         stop-service HostControllerService

·         Get-MailboxDatabase <DBName> | select EdbFilePath

·         Navigate to the directory

·         Delete the guid path under the same dir

·         Set-MailboxDatabase “<database name>” -IndexEnabled:$False

·         start-service MSExchangeFastSearch

·         start-service HostControllerService

 

Step 5 – Finishing off mailbox migration

§  Once you are at the point where every move request is in the state “autosuspended”, you are ready to complete

§  Suspend mail flow into your organisation, so waiting messages will get queued

§  Get-moverequest | set-moverequest –suspendwhenreadytocomplete:$false

§  Get-Moverequest | resume-moverequest

 

 

Step 6 – ADMT

·         We can now use ADMT to migrate SIDHistory and group memberships into the target domain

·         Start ADMT

·         User account migration wizard

o   In step 3 we grabbed a list of all mailbox alias  – Get-mailbox | select alias >>C:\temp\org1.txt

o   We are going to use this list as the list of accounts we need to migrate (as we only want to migrate users with mailboxes)

o   There is an issue with this method in that sometimes the mailbox alias will not match the username

§  This generally occurs when a user has changed their name (such as when someone gets married) and the admin was a little bit lazy and didn’t update it everywhere

§  I’ve also found some instances where admins change the mailbox name of shared mailboxes as the business requests, but doesn’t change other items, such as the alias, username etc

§  ADMT will help you identify these accounts in a later step

·         Select your source and destination domain

o   If you are suing password export, the DC selected must be the DC with the PES (pass word export server) installed

·         Read objects from an include file

o   Select :\temp\org1.txt (or org2 as appropriate)

·         At this point you will get an error popping up if any of the accounts specified cannot be found in the source domain…. At the point you have two options

o   Go back and fix the source account info – this is what I recommend – however, if you are changing usernames, you obviously may need to notify users (although if its for a resource mailbox that no-one directly logs into, you will not)

o   Modify the text file to match the names of these accounts

·         Select a target OU

·         Select if you wish to migrate passwords or generate complex passwords

·         Select the target account state (if this is resource forest, the user accounts should be disabled in the destination domain)

·         Most importantly – ensure “Migrate users SID to target domain”

·         Enter the username and password

·         Away you go

 

Step 7 – migrating DL’s

·         You will already have DL’s created and populated, thanks to the ADMT in step 6, however these groups will be lacking mail enabled-ness and SMTP addresses

·         This article – http://blogs.technet.com/b/sachinf/archive/2014/03/17/how-to-migrate-distribution-groups-across-a-forest.aspx – contains some details on using some Microsoft provided scripts for migrating DL’s… but no great surprise being MS…. It does the job, but, well, it doesn’t make it easy

·         Download the toolkit from http://blogs.technet.com/b/exchange/archive/2012/02/03/released-migrating-from-exchange-server-2010-in-hosting-mode-to-exchange-server-2010-sp2-whitepaper.aspx

·         Extract the lot, and then go to the directory \HostingMode_to_SP2_SampleMigrationScripts

·         Copy this directory to your source and target domains

o   .\QueryCustomerOrg.ps1 <domain name>

o   Edit the following

§  $PreferredDC = <DC FQDNS>

§  $DomainID = <NetBIOS name>

§  $DomainSuffix = <domain suffix>

§  $HostingOU = <OU>

§  $BaseSearchPath = <DNS of top level>

§  $OutputFile = “C:\Scripts\ ”                                         <if you don’t plan on running from C:\scripts, change this>

§  $xml = [xml](get-content “C:\Scripts\”)                 <if you don’t plan on running from C:\scripts, change this>

§  $searcher.SearchScope = “Subtree”                        Change to Subtree if you want to the script to be of any use

o   The script must be run with an argument of the OU to check, so simply enter your OU (this is why you put the top level in the $BaseSerachPath)

o   Run the script

§  Then run CreateADMTGroupList.ps1 <source to xml generate above> <NetBIOS domain name>

§  Copy the xml and group list text to the destination domain

o   On the destination server

§  Edit Restoregroupattributes.ps1 with your DC name

§  Run the script .\ResotreGroupAttributes.ps1 <path to xml file>

o   Rinse and repeat for all other domains you are migrating

·         In my move from 2010/2013 to a 2013 resource forest, I found a few issues with this script

o   Not all groups got converted to mail-enabled universal groups

o   Other properties of the groups, such as delivery management were not copied

 

Step 8 – Fixing Issues

I had the following issues in my migrations:

·         Auto-mapping of mailboxes where the user has full permission no longer auto-mapped

o   This was a factor of two things

§  The attribute MSexchDelegateLinkList was not migrated on the shared mailboxes. It was a relatively short list for my migrations, so I did this manually

§  One of the “old” domains, which was no longer in use, did not have autodsicover pointed to the new box. This seemed to break auto-mapping where the shared mailbox was an “old” mailbox which was only used for reference purposes

·         The client used airwatch to manage mobile devices

o   This updated settings, but required users to re-enter their password

o   As you can imagine, for some users, this was fine…. For others, the sky had fallen/it was the end of the world

o   I left this to the internal team to manage, and I probably should have been more involved and understood the impacts

·         One account had a Sid in the “linked account” instead of the account (not sure why), needed to be re-updated using

o   Get-mailbox -identity <name> | set-mailbox linkedmatseraccount <linked account name> -linkedodmaincontroller <dc name in source domain>

 

Appendix – things I would do before my next Exchange inter-org migration (if one ever comes up again)

·         Users can “favourite” shared contacts from public folders, this does not stick once those PF’s are migrated to a new exchange org.

o   Write a script to discover this setting, and re-instate it after migration

·         Write a script to export “MSExchDelegateLinkList” and import it within the new forest to address mailbox auto-mapping

·         Write a script to migrate contacts

·         Write a better script to migrate DL’s from forest to forest, the one in step 7 is intended for 2010, so misses some 2013 attributes – and is quite clunky

2 thoughts on “Exchange inter-forest migration tips

  1. Hayes, We are facing this same task in a few months. Two separate orgs now into one. Two Exchange 2010 orgs. We have a new forest going but just at the beginning of that. No users or anything migrated. Which brings me to my question… Did you have all your users, groups, computers and other resources migrated over the the new forest before the Exchange parts? Or in what order did you start the migration of old forests into the new one? Thanks! Enjoyed your explanation a lot.

    1. Sorry for the slow reply mike…. i have been doing this whacky thing called “focusing on life rather than work” for a few months now…. so i havent been checking the blog regularly.

      So – assuming you havent worked this out for yourself already (you probably have) – yes, i use the ADMT to migrate all my users (at a minimum) over to the destination forest first. Keep in mind that you can run and re-run the ADMT multiple times, and as such, you can migrate certain objects without having to migrate everything at once. Even once you have done a bucketload of interforest migrations, its really handy to be able to run and re-run so you can test different scenarios or parts of the migration you may not be 100% on.

      Again – sorry for the late reply!