What is the maximum Mailbox Database size?
16 TB
However, this is not recommended, it is merely the theoretical maximum that it can support.
A smaller size is recommended to avoid issues with backup, restore, repair, recovery, and performance.
What is the recommended maximum mailbox database size?
200 GB for non-replicated databases
2 TB for replicated databases.
What is JBOD?
Just a Bunch of Disks
An arrangement of a single physical disk, or multiple disks which are not using RAID, and have no physical redundancy or parity.
When using JBOD, you need to use application-level replication for redundancy,
What is DAG?
Database Availability Group
The mechanism of high availability for Exchange 2016.
When does Microsoft recommend using JBOD?
Microsoft recommends JBOD for Exchange deployments that use DAG, for the database and log volumes (but not the OS volume).
When is NTFS vs. ReFS recommended?
Use NTFS for the OS volume, and the Exchange application volume (which is typically the same volume). ReFS is not compatible.
Volumes storing databases and transaction log files may use either NTFS or ReFS, but ReFS is recommended because of the benefits.
What are the benefits of ReFS?
It’s more resilient.
- Enhanced data verification and auto-correction
- more resilient to file system corruption.
- This is particularly true with volumes hosting large files. (and Exchange DB files are usually large.)
How should ReFS be configured for Exchange?
You should disable integrity streams on Exchange DB and Log volumes, for performance reasons.
This does not disable the benefits of ReFS, and it avoids a serious performance hit.
The GUI can format as ReFS, but does NOT HAVE an option to disable integrity streams, so you must use PowerShell.
Provide an example PowerShell command for formatting a newly added disk meant to store an Exchange database.
(Answer has line breaks inserted for readability.)
Get-Disk 5 | Initialize-Disk -PartitionStyle GPT -PassThru | New-Partition -UseMaximumSize | Format-Volume -FileSystem REFS -NewFileSystemLabel Volume1 -SetIntegrityStreams $false
What is Auto Reseed?
A feature of Exchange 2016 DAGs that automatically detects and recovers from storage failure scenarios.
When a disk fails in a DAG and the Auto Reseed process is triggered, what happens?
Auto Reseed will bring the DAG into a fully healthy state by:
- automatically mounting a disk from a pool of spares (that were preconfigured),
- formatting that disk for use,
- and reseeding the database copies onto the new disk.
What are the storage layout requirements for Auto Reseed?
• JBOD, not RAID
• Multiple databases per volume.
– Number of DBs should equal number of copies of each DB. (4 member DAG should have 4 DBs per volume.)
• Mount points, not drive letters
What will trigger Auto Reseed?
- MSExchange Replication Service checks for database copies that are FailedAndSuspended
- If all database copies on a volume are FailedAndSuspended for 15 minutes, then the Auto Reseed recovery workflow is started.
What configurations are recommended when virtualizing an Exchange server?
- Max of 2:1 ratio of vCPU to logical CPU (1:1 preferred)
- Size the server using MS’s sizing calculator, which adds overhead for virtualization
- Fixed memory allocation (no Dynamic Memory)
- Full storage allocation (no Thin Provisioning)
What common virtualization features are not supported with Exchange?
- Any “hot” migration or failover mechanism that doesn’t start the VM on a new host as a cold boot, except for Hyper-V Live Migration
- Any replication-based disaster recovery solution (MS recommends using DAGs for replication)
- Any use of VM snapshots to “roll back” an Exchange VM to a previous state.
What is Jetstress?
A Microsoft tool that simulates Exchange 2013 and Exchange 2016 disk I/O load on a server to verify the performance and stability of your disk subsystem before putting your server into a production environment.
Jetstress validates that the storage you configure meets the requirements Exchange will have.
When should Jetstress be run?
- On a server before installing Exchange.
- If using RAID volumes, test when volume is both healthy and degraded (because there’s no point to RAID if the volume becomes unusable when degraded)
- If using shared infrastructure (e.g. virtualization), test during peak load times
How does Jetstress work?
Simulates disk I/O load for a given number of users that you specify
and in doing so, it tests:
- Disk hardware
- Storage firmware
- Storage controllers
- File system
How large can a DAG be?
A DAG can consist of up to 16 mailbox servers, plus a FSW server.
What is a FSW?
File Share Witness
Every DAG has one FSW.
A non-DAG member of the DAG, running Windows Server.
It serves as a tie-breaker during quorum voting for DAGs with an even number of members.
How many databases can an Exchange 2016 Standard edition server host?
A maximum of 5 databases
How many databases can an Exchange 2016 Enterprise edition server host?
A maximum of 100 databases
What are the OS and software versioning requirements for DAG members?
DAG Members must:
- All run the same version of Windows Server
- All run the same major version of Exchange:
○ A mix of different CU versions is ok.
○ A mix of Standard and Enterprise editions is ok.
○ But cannot mix different major versions, such as 2013 and 2016.
What is a Database Switchover?
An administrator (human) -initiated move of active database copy.
The database copy that was previously the “active copy” is dismounted and becomes a passive copy.
Then, another copy that had previously been passive is mounted and becomes the active copy.
How are database copies utilized within a DAG?
For any given database, one DAG member has a copy of it which is designated as the “active copy” of that database. A database only has one active copy at a time.
The active copy is the one that is mounted and servicing clients.
Passive copies exist on other DAG members, are not mounted, and are updated via continuous replication.
You can have up to 15 passive copies of a database (across 15 other DAG members.)
What does it mean to “Activate” a database copy?
Switch it from being a passive copy to the active copy (and simultaneously dismounting the previously active copy to make it passive).
It’s another term for a Database Switchover.
What is a Database Failover?
A system-initiated (automatic) move of active database copy in response to a fault, such as a disk or server failure.
Essentially, the same thing as a Database Switchover, except it happens automatically.
What is a CAAP?
Cluster Administrative Access Point
A computer account with a single IP address that serves as an administrative access point for a DAG.
Exchange 2016 can use a CAAP, however, it defaults to not using it.
What are the disadvantages of using CAAP?
It makes management a little more complicated, and introduces two failure domains: a potential for IP address conflict, and a computer account which could be disabled or deleted.
When setting up a dedicated DAG replication network, what settings are required for Auto-Configuration to work?
- Do not configure a DNS server or Gateway on the Ethernet Adapter.
- In TCP/IP Settings > Advanced > DNS > uncheck “Register this connection’s address in DNS”
- Disable “File and Printer Sharing for Microsoft Networks”
- Disable “Client for Microsoft Networks”
Provide an example PowerShell command for adding an Exchange Server to a DAG.
(Answer has line breaks inserted for readability.)
Add-DatabaseAvailabilityGroupServer
- Identity
- MailboxServer
How many IOPS will a database copy have when it is a passive copy, in comparison to what it will have as an active copy?
About 50%
What is an Activation Preference?
A property of database copies.
It indicates the administrator’s preference for which database copy is most prefered to be active.
It is considered by Active Manager during failovers and targetless switchovers, but it is not the only deciding factor.
A database will have a preference number set for each server that hosts a copy of it. For example, a DAG with four members will have a preference of 1, 2, 3, and 4 assigned to each server for each database.
What is a targetless switchover?
An manually (administratively) initiated event where a database copy is changed from active to passive, and another database copy will become active, without specifying which copy to activate.
It’s a Database Switchover, except the database copy to activate is not specified.
Active Manager will determine which copy to activate.
What is the “PreferenceCountList” property of a DAG member, and how is its data formatted and understood?
It shows how many databases it has set to different Activation Preference levels.
It is displayed in this format:
{2, 3, 0, 1}
The above example indicates that this server has:
- 2 database copies with Activation Preference 1
- 3 database copies with Activation Preference 2
- 0 database copies with Activation Preference 3
- 1 database copies with Activation Preference 4
How can Azure be utilized for multi-site DAGs?
You can have a File Share Witness (FSW) in Azure as a “third-site” FSW, which provides a quorum no matter which other site may fail.
What is DAC Mode?
Datacenter Activation Coordination Mode
A property of DAGs designed to prevent split-brain conditions when DAG members become isolated.
What is DACP?
Datacenter Activation Coordination Protocol
Used by DAC Mode in DAGs.
How does DACP work?
- Each DAG member has a “DACP Bit” which is set to “0” upon startup.
- It won’t mount a database copy until the DACP bit is set to 1.
- DACP only changes to a “1” if the DAG member can communicate with another DAG member with a DACP bit of 1, or if it can communicate with all other DAG members.
How do you enable DAC Mode?
PowerShell command
Set-DatabaseAvailabilityGroup
- Identity
- DatacenterActivationMode DagOnly
When is it recommended to use DAC Mode?
Recommended:
Enable DAC Mode for all DAGs with two or more members, even if the DAG is only within a single site.
The only exception is for DAGs using third-party hardware replication.
What is a Content Index?
Each Exchange database has a content index, which is used for searches by:
- Outlook on the web
- Online-mode Outlook clients (non-cached mode clients)
- Also used for eDiscovery searches.
What is a Catalog?
Another name for a Content Index
What is BCSS?
Best Copy and Server Selection
The process used by Active Manager to choose which database copy is activated in a failover event.
The goal is to failover a database to the healthiest passive copy running on the healthiest server.
What are some of the variables considered by Active Manager in BCSS?
- AutoDatabaseMountDial
- Activation Preference
- Copy Queue Length
- Replay Queue Length
- Database copy health
- Server health
What is CQL?
Copy Queue Length
A value of a database copy, indicating the number of transaction log files that have yet to copy across the network to this database copy.
What is the “AutoDatabaseMountDial”, and what can it be set to?
A setting on Exchange mailbox servers that can have one of three values:
- GoodAvailability (the default setting)
- BestAvailability
- LossLess
What does this setting of the AutoDatabaseMountDial do?
GoodAvailability
Allows a database copy to be auto-activated by Active Manager if it has a CQL of 6 or less.
Though there is some small risk of message loss, this setting prioritizes maintaining availability.
Additionally, data loss due to missing log files is mitigated by Safety Net.
What does this setting of the AutoDatabaseMountDial do?
BestAvailability
Allows a database copy to be auto-activated by Active Manager if it has a CQL of 12 or less.
Though there is some small risk of message loss, this setting prioritizes maintaining availability.
Additionally, data loss due to missing log files is mitigated by Safety Net.
What does this setting of the AutoDatabaseMountDial do?
LossLess
Allows a database copy to be auto-activated by Active Manager only if there are NO missing transaction log files (i.e., the CQL must be 0).
Though there is no risk of data loss, you risk the database going offline if replication is not 100% healthy and up-to-date at the time of failure.
What is Safety Net?
A cache of e-mail that has already been delivered to mailboxes.
When a “lossy” failover occurs, Exchange servers are able to request that the missing messages be resubmitted from Safety Net.
What is a “Soft recovery”?
Using ESEUtil, it repairs a corrupted/failed database without any data loss, by rebuilding data from the transaction logs.
This requires that all transaction logs be available and healthy.
What is a “Hard recovery”?
Using ESEUtil, a risky repair operation for a corrupted database that may result in data loss.
It should only be run on damaged databases
It does not apply any information in the transaction log files to the database.
Essentially, it goes through the database, and simply deletes any damaged or corrupt pages.
It may result in data loss, possibly small, possibly large, depending on the amount of corruption.
It should only be performed as a last resort. You should even contact Microsoft Support before doing this, to see if there is any better option.
What options are there when a database copy fails / becomes corrupted?
If running a DAG, the best option is to reseed the database from a healthy copy, rather than repairing it, if possible.
For databases with a single copy, or no healthy copies, use ESEUtil to repair it:
You can do either a “soft repair” (the safer option, with no data loss), or a “hard repair” (risks data loss, use only as last resort).
What options are there when a database copy fails / becomes corrupted?
If running a DAG, the best option is to reseed the database from a healthy copy, rather than repairing it, if possible.
For databases with a single copy, or no healthy copies, use ESEUtil to repair it:
You can do either a “soft repair” (the safer option, with no data loss), or a “hard repair” (risks data loss, use only as last resort).
What is the Powershell command for a soft recovery?
eseutil /r /l /d
Example:
eseutil /r EO4 /l E:\Logs /d D:\Database
What is the Powershell command for a hard recovery?
eseutil /p
Example:
eseutil /p SingleDB.edb
What is RPO?
Recovery Point Objective
Defines the point to which data must be restored, or how much data loss the organization is willing to accept, in a recovery scenario.
Example RPO = 4 hours
What is RTO?
Recovery Time Objective
The amount of time in which the recovery must be completed.
Example RTO: 30 minutes
What does a “Full backup” do, and what is needed to restore to the most recent Full backup?
- Backs up all data
- Truncates logs
- To restore, just need the most recent full backup.
What does an “Incremental backup” do, and what is needed to restore to the most recent Incremental backup?
- Backs up all data that has changed since the last full or incremental backup
- Truncates logs
- To restore, you need the most recent full backup plus all incrementals since.
What does a “Differential backup” do, and what is needed to restore to the most recent Differential backup?
- Backs up all data that has changed since the last full backup
- Does NOT truncate logs
- To restore, you need the most recent full backup plus the most recent Differential.
What is the “dumpster”?
When a “Deleted Items” folder is emptied, or items within it are deleted, they go to the “dumpster.”
Items in the dumpster are recoverable by end-users via the “Recover Deleted Items” option that appears in Outlook (both in the application and on the web).
When are items removed from the dumpster?
- Dumpster items deleted automatically after 14 days by default, but this retention setting can be modified.
- Additionally, end-users can manually purge their dumpster at any time.
What is the PowerShell command to modify the length of retention of Deleted Items on a database?
Set-MailboxDatabase -Identity YourDB01 -DeletedItemRetention 30.00:00:00
(format is dd.hh:mm:ss)
What is SIR?
Single Item Recovery
An option that, when enabled, when retain deleted items from a mailbox until the set retention period has passed, even if a user purges their dumpster.
- It is Off by default
- It is set on a per-mailbox basis
How are SIR items recovered?
- They are recovered using the Search-Mailbox cmdlet.
* They cannot be recovered by end-users in Outlook once they have purged their dumpster.
What PowerShell command enables SIR?
Set-Mailbox -Identity John.Smith@contoso.com -SingleItemRecoveryEnabled $true
How long will SIR retain deleted and purged items for?
By default, it will inhereit from the host Database’s retention policy. But, it can be modified on a per-mailbox basis.
What PowerShell command adjusts SIR retention settings?
Set-Mailbox -Identity John.Smith@contoso.com -RetainDeletedItemsFor 90.00:00:00
(This presumably also modifies the dumpster retention setting for that user.)
What is this:
GUID
Globally Unique Identifier
A code ID number given to some database and directory objects.
How can a mailbox be recovered (without using a backup)?
- Disabled mailboxes are retained in the database for 30 days by default.
- A disabled mailbox can be reconnected to the same user object, or a different user object.
- The contents can also be restored to another mailbox.
What do Disable-Mailbox and Remove-Mailbox commands do?
Both commands will mark the mailbox, and any archives present, for removal in the database.
Once marked, it will be deleted after a set retention period, which is 30 days by default.
In addition:
- “Removing” the Mailbox will also remove the AD user object.
- “Disabling” the Mailbox merely removes the Exchange properties from the AD user object.
What steps are required to Backup Exchange using Windows Server Backup?
1) Install Windows Server Backup, as it is not installed by default.
2) You may want to select a Custom backup, rather than Full server, since you really only need to backup the database and log volumes, rather than the OS and Application volume.
3) Go into Advanced Settings > VSS Settings, and be sure to select “VSS Full Backup,” since this will truncate the log files. (VSS Copy Backup will not.)
In a disaster, what needs to be recovered for Exchange to be restored?
And, how should they be restored?
• Operating System
- Should be reinstalled and rebuilt rather than restored from backup.
• SSL certificates
- Import them again
• Exchange 2016 application files and services
- Perform a Recovery Install of Exchange 2016
• Exchange 2016 mailbox databases
- Restore from backup, or, if in a DAG, reseed from another DAG member server.
In a recovery scenario, why should you not restore an entire Exchange server from backup?
This would be similar to rolling back a VM to a previous state.
Exchange does not support
“time-travel” / being rolled back to a previous state.
You could potentially do significant damage to the Exchange Application, Exchange Data and Active Directory.
What recovery options will work for recovering a Mailbox Database using Windows Server Backup?
Using the Recovery Wizard, you have three recovery options:
1) Files and folders
2) Volumes
3) Applications
For recovering a Mailbox Database using Windows Server Backup:
What is the procedure for using a “File and folder” recovery?
The database recovered from backup will be in a “dirty shutdown” state, so you will need to manually recover the database using eseutil.exe so it can be mounted.
(Same as with a “Volume” Windows Server Backup recovery)
For recovering a Mailbox Database using Windows Server Backup:
What is the procedure for using a “Volume” recovery?
The database recovered from backup will be in a “dirty shutdown” state, so you will need to manually recover the database using eseutil.exe so it can be mounted.
(Same as with a File and folder” Windows Server Backup recovery)
For recovering a Mailbox Database using Windows Server Backup:
What is the procedure for using an “Application” recovery?
This is the best option for recovery from Windows Server Backup, because:
Exchange is a registered application, so the restore can “roll forward” the transaction logs that are still found on the transaction log volume, to bring the database back up to the time at which it failed, rather than merely to the time of the last backup.
This roll-forward is optional if not desired.
Note, there is no way to select specific databases in the restore, so you will need to restore all Exchange databases contained in the backup, even if you only need one of them.
What does installing Exchange using Recovery Mode do?
The installation will use the Exchange server configuration that is still stored in Active Directory from before the server failed.
How do you install Exchange in Recovery mode?
It can only be performed from command line.
Using the drive letter that the Exchange installation media is mounted to:
E:>setup /m:recoverserver /Iacceptexchangeserverlicenseterms
How would you restore a failed Exchange server that is a member of a DAG?
Besides configuring the OS, storage, and other prerequisites:
- In the DAG’s configuration, remove the Database Copies that were configured for the failed server
- Remove the failed server from the DAG
- Install Exchange using Recovery mode.
- Evict the failed server from the Windows Failover Cluster (which underlies the DAG).
- Add the server back into DAG membership
- Add the DAG database copies back to the server
- Reseed the database copies
- Reconfigure/rebalance Activation Preferences, since they will default to the lowest preference for the most recently added server.
What is eseutil?
“Extensible Storage Engine Utilities for Microsoft Exchange Server”
What is a Recovery Database?
A Mailbox Database that has been restored from a file backup, placed in an alternate path, and mounted for recovery purposes only.
They are useful if other recovery options have already passed their retention period.
How do Recovery Databases impact Standard Edition Exchange’s Database limit?
Recovery Databases do not count toward the limit of 5 databases for Standard Edition.
What can be done with a Recovery Database?
Clients cannot connect to mailboxes stored in mounted recovery databases.
They can only be accessed by admins for the purpose of performing restores.
How do you recover Public Folders?
Use the same options and steps as other mailbox types.
But additionally, if the public folder contained mail-enabled folders before it was deleted:
1) Make note of SMTP addresses on reconnected mailbox
2) Mail-disable the public folder mailbox
3) Mail-enable the public folder mailbox
4) Re-apply the missing SMTP addresses
What may happen when recovering a Primary Hierachy Mailbox?
Restoring a primary hierachy mailbox to a point in time may cause folders created in the hierarchy since that time to be deleted.
As such, be sure to protect the primary hierachy mailbox and AD user object from accidental deletion.
What is a Lagged Database Copy?
A passive database copy that has been configured with a delay before transaction logs are replayed.
This provides the ability to recover the database from an earlier point in time without traditional backups.
What steps are required for configuring a Lagged Database Copy?
- Apply a replay lag interval to the database copy.
- Suspend the database copy from automatic activation
- Ensure Safety Net is configured with at least the same time interval.
Provide an example PowerShell command to configure the Safety Net hold time.
Set-TransportConfig -SafetyNetHoldTime 10.00:00:00
format is dd.hh:mm:ss
What is a Dial Tone Database, and how is it used?
When a mailbox database fails, a new, empty database can be mounted temporarily, allowing users to send and receive e-mail while the failed database is in the process of being recovered. This is called a “Dial Tone Database.”
Once the failed database is recovered:
- The Dial Tone database is dismounted,
- the recovered database is mounted as the primary database, and
- the Dial Tone Database is mounted as a recovery database, and its contents are restored into the active database, thus merging them.
