At Portiva we like to use ShareGate for migrating SharePoint data. ShareGate offers a clear and user-friendly application to kick off a migration. But what if you have so many sites to migrate that manually starting migrations takes a long time? In this blog you will learn how to easily automate your migration jobs with the ShareGate Powershell module and Windows Task Scheduler.
ShareGate and the PowerShell CLI
Before we can get started creating the necessary code, we need a few things:
• PowerShell (minimum version 5)
• SharePoint Online module (Install-Module SharePointPnPPowershellOnline)
• ShareGate installed and activated on a Windows machine
• A SharePoint list where we store the migration items, also called the migration list
• The demo code (found here). This contains the code examples we will be using in this blog
To create a PowerShell script that can use ShareGate, the ShareGate PowerShell module must first be imported. Then three connections will be created; 2 connections are needed to actually migrate data from the old environment (connection 1) to the new environment (connection 2), a third connection is needed to keep the SharePoint list up-to-date with the migrated items statuses. To keep the script as clean as possible, the credentials of these connections have been created in the file "variables.ps1".
We then retrieve the ongoing tasks that ShareGate is currently still working on. This allows us to start a maximum number of tasks and minimizes the risk of throttling. The maximum number of migrations we can start is therefore "maximum running tasks minus current tasks". If fewer than the maximum number of tasks are running, we retrieve the maximum number of migration items from the migration list using the "Connect-PnPOnline" and "Get-PnPMigrationItem".
Then we iterate through the list of migration items and set some variables that we assign to the desired migration actions. In our example, this is mass migration of a site, starting a delta migration, or copying a Team.
Note: ShareGate version 13.x is required to copy a Team.
The migration script connects to the old and new environment and starts the ShareGate task. Using the Set-PnPListItem, we save the new status of the migration item.
Automated ShareGate migrations
When we run the script in PowerShell, the maximum allowed tasks will start running. However, this will not initiate the next migrations by itself. To start the script automatically, in timed intervals, we use the Windows Task Scheduler application.
Create a new task and set a desired interval for checking running tasks.
Create a new action that runs the Start-Migration PowerShell script.
Use the path to PowerShell.exe, for example:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Under ‘Add Arguments’ fill in the following:
-noprofile -executionpolicy unrestricted -noninteractive -File "Path_To_PSFile"
This ensures that PowerShell runs in the background and does not request permission to run certain commands.
Save the job and wait for the next timed trigger. And voila; ShareGate will now be controlled every x minutes using the Task Scheduler and the PowerShell scripts.
Pro Note: To make sure the PowerShell code isn’t terminated until all jobs are done, the following code is added in the Start-Migration script. Running the migration script without this can make PowerShell close itself before all background jobs are finished.
$runningBackgroundJobs = Get-Job -State Running
while($runningBackgroundJobs.Count -gt 0){
Start-Sleep -Seconds 1
$runningBackgroundJobs = Get-Job -State Running
}
Conclusion
With this demo script it is easy to start your automated migration script. It just lacks one important feature: proper logging. The Start-Transcript does allow us to see some of the logging, but it is advised to create a logging system to log all possible errors, enable try-catching and store the logging in a convenient place.
Anyways, I hope this blog helps you to kick-off your automated ShareGate migrations. If you have any suggestions, questions or problems feel free to contact me on Twitter (@KnijnOps) , or via email.
