Event ID 5003 Generated When Starting Exchange 2010 Information Store Service (Even With Correct Server Time)

I ran across an interesting issue with an Exchange 2010 Mailbox Server (SP1 RU6) recently where the Information Store service would not start on a DAG member after a reboot. Every time I tried, even after additional reboots, I got this error message:

Source: MSExchangeIS
Event ID: 5003
Description: Unable to initialize the Information Store service because the clocks on the client and server are skewed. This may be caused by a time change either in the client or the server, and may require a reboot of that computer. Verify that your domain is properly configured and is currently online.

I checked the clock and compared it to the domain controllers in the site as well as the other DAG members, they all matched within a second of each other (technically you should not receive an error like this unless the difference is greater than five minutes).

As we run an NTP server on our domain, all of the clocks are automatically synchronized everywhere so it was a bit puzzling whywe woudl receive such an error.

Finally, I caught on to what was going on. When the server was booting, it was initially setting its clock according to the BIOS (which in this case was provided by the VMWare ESX host) before synchronizing with the domain. After consulting with the engineering team that manages the ESX farm here, the host this particular Exchange server was on did not have time synchronization configured, and it had fallen out of sync with the rest of the network by seven minutes.

Moments after booting up, the clock on the Exchange server would be reset to the domain time, but by then the Information Store service had already tried, and failed, to start. It appears there is some sort of “feature” that then prevents it from starting even after the time is corrected.

The fix was simply to fix the time on the ESX host (as well as configure NTP on that host) so that the time would be correct on boot up. After rebooting the Exchange server again, the Information Store service started up without any problems.

Advertisements

About Rebecca Harness

Rebecca Harness is a Business Information Security Officer (BISO) for a publicly-traded, global information solutions company. As BISO, she champions security initiatives and recommends strategies to mitigate risk, facilitating innovation and new product development. She’s also responsible for representing the business unit’s security program in client facing engagements, conferences, and industry forums. Prior to her current role, she was an influential cybersecurity leader for one of the world’s largest transportation providers, known for transforming information security efforts into well-orchestrated programs. There, she developed an innovative methodology for delivering key information security priorities as a service model, leading to quicker adoption enterprise-wide while significantly reducing operational costs. She also led and modernized their global, multi-brand PCI Assessment and other compliance initiatives. In the early 2000’s, Rebecca developed one of St. Louis' leading Managed Services Providers from a startup in a spare bedroom into a mature consulting company with 30+ employees and 150+ clients in the Greater St. Louis Area. Rebecca holds many certifications, including; ISACA Certified Information Systems Auditor (CISA); ISC2 Certified Information Systems Security Professional (CISSP); and GIAC Security Leadership Certified (GSLC). She’s also a proud alumni of Hastings College and a longtime member of the Society of American Magicians.

11 Responses to “Event ID 5003 Generated When Starting Exchange 2010 Information Store Service (Even With Correct Server Time)”

  1. This tip really helped me out with an identical issue.

    Thanks,

    Martin

  2. Great catch, i had the very same issue after NTP stopped working at my DR site. I really couldn’t understand how i was getting these errors when the clocks seemed in sync. Thanks!

  3. Many thanks.
    I had the very same problem but the “vm sync with ESX” was UNCHECKED.
    The ESX (vsphere 5) had 1h less than the real time…very strange
    why the VM still sync with ESX regarless of the time option ?!

  4. This post saved me after a disastrous SP2 rollback! Still getting through the garbage left over, but at least the store will start up. Thanks much.

  5. +1. Who would have thought the host time had any affect on the guest. Simple resolution… once you know what to look for.Great info; thank you!

  6. +1, Thanks for the comment similar problem here: time on the machine was on a perfect sync win my dc’s but the esx host was not. NFS service on esx was stoped after an upgrade from 5.0 to 5.1

  7. Great Info! Had a similar issue this evening. After migrating the VM to a host with the correct NTP server: IS service stayed started and the DB mounted.

  8. Thank you!! This was killing me for a while. The clock was 9hrs off on VMWare ESX Host! The first time around the clock was off on my Exchange 2010 server, then when I sync’d to domain, each reboot would cause the message again even though the time looked right after it was completely rebooted.

    Thanks again, you’re a life saver.

  9. That was a life saver. My ESXi Clock was only off by seven minutes but after correcting everything booted no problem. Thanks!!!

  10. Thank you for this article. Our ESXi Clocks were off by 8 minutes. After correcting, everything worked like a champ.

  11. thank you for sharing this experience. I got exactly the same.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s