Failed Database Content Index for Exchange 2013

Note: this tip applies to Exchange Server 2013 Mailbox servers that are not members of a Database Availability Group. For failed content indexes on DAG members refer to this article.
On an Exchange Server 2013 server you may encounter failed content indexes that are preventing end users from being able to run searches in OWA and Outlook.
A failed content index will be visible in the output of Get-MailboxDatabaseCopyStatus:



[PS] C:\>Get-MailboxDatabaseCopyStatus * | ft -auto
Name             Status  CopyQueueLength ReplayQueueLength LastInspectedLogTime ContentIndexState
---- ------ --------------- ----------------- -------------------- -----------------
DB01EX2013SRV1  Mounted 0 0 FailedAndSuspended
DB02EX2013SRV1  Mounted 0 0 FailedAndSuspended
DB03EX2013SRV1  Mounted 0 0 Healthy
DB04EX2013SRV1  Mounted 0 0 FailedAndSuspended
DB05EX2013SRV1  Mounted 0 0 Disabled
In the example above the content indexes for DB01, DB02, and DB04 are failed.
Other indications of a problem can be seen in the Application event log, for example:




Log Name: Application
Source: MSExchangeIS
Date: 2/16/2015 11:09:26 AM
Event ID: 1012
Description:
Exchange Server Information Store has encountered an error while executing a full-text
index query ("eDiscovery search query execution on database 191987bf-5e9f-4ba4-b13b-3cadcb9e51f5
failed."). Error information: System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]:
Internal error while processing request (Fault Detail is equal to An ExceptionDetail, likely created by
IncludeExceptionDetailInFaults=true, whose value is:
Microsoft.Ceres.InteractionEngine.Component.ProcessingEngineException: Internal error while processing request
at Microsoft.Ceres.InteractionEngine.Component.CieProcessingEngine.LogAndRethrowException(Exception e)
Log Name: Application
Source: MSExchangeFastSearch
Date: 2/16/2015 11:06:13 AM
Event ID: 1009
Description:
The indexing of mailbox database DB02 encountered an unexpected exception. Error details:
Microsoft.Exchange.Search.Core.Abstraction.OperationFailedException: The component operation has failed.
---> Microsoft.Exchange.Search.Engine.FeedingSkippedForCorruptionException: "Feeding was skipped for
'63fe7551-8100-4e3e-9a3e-4b14744eddb6 (DB02)' due to the state 'Failed', error code: 'CatalogCorruption',
failure code: '2400519', failure reason: 'Failed to initialize FastServer: Generation mismatch:
0 < GID[82381] [IndexName=63FE7551-8100-4E3E-9A3E-4B14744EDDB612.Single]'."
at Microsoft.Exchange.Search.Engine.SearchFeedingController.InternalExecutionStart()
at Microsoft.Exchange.Search.Core.Common.Executable.InternalExecutionStart(Object state)
--- End of inner exception stack trace ---
at Microsoft.Exchange.Search.Core.Common.Executable.EndExecute(IAsyncResult asyncResult)
at Microsoft.Exchange.Search.Engine.SearchRootController.ExecuteComplete(IAsyncResult asyncResult)
To rebuild a failed content index we first need to stop the search services on the Exchange server. Note that this may impact searches for other healthy databases, and the rebuilding process can also create a significant load on the server, so you may wish to do these steps outside of normal business hours.
Stop the following services:
Microsoft Exchange Search Host Controller

Microsoft Exchange Search




[PS] C:\>stop-service MSExchangeFastSearch
[PS] C:
b62
\>stop-service HostControllerService
Navigate to the location of the content index for the database. This will be the same folder that the database file is located in. For example, DB01 is located in F:DB01.




[PS] C:\>Get-MailboxDatabase DB01 | select EdbFilePath
EdbFilePath
-----------
F:DB01DB01.edb
The content index is stored in a folder named for the GUID of the database.



Delete the folder. Repeat the same steps to delete the folder for any other failed content indexes you're also dealing with at the time.
Then start the search services again.

[PS] C:\>start-service MSExchangeFastSearch
[PS] C:\>start-service HostControllerService
The content indexes will be rebuilt, which can take quite a while to complete depending on the amount of data in the databases.
Eventually you should find that your content indexes are healthy again.

[PS] C:\>Get-MailboxDatabaseCopyStatus * | ft -auto
Name             Status  CopyQueueLength ReplayQueueLength LastInspectedLogTime ContentIndexState
---- ------ --------------- ----------------- -------------------- -----------------
DB01EX2013SRV1  Mounted 0 0 Healthy
DB02EX2013SRV1  Mounted 0 0 Healthy
DB03EX2013SRV1  Mounted 0 0 Healthy
DB04EX2013SRV1  Mounted 0 0 Healthy
DB05EX2013SRV1  Mounted 0 0 Disabled