Exchange 2016 Hybrid : TLS negotiation failed with error UnknownCredenta


I was adding couple of Exchange 2016 servers with CU2 to the Hybrid configuration wizard to send and receive emails to Exchange Online. On Exchange Online Admin center, I configured the receive connector to Office 365 o verify the subject name on the certificate for TLS authentication.

The problem is that emails are not being sent to Office 365 via the send connector. After enabling the protocol logging on my Exchange 2016 hybrid servers [Get-SendConnector “outbound to Office 365” |Set-SendConnector -ProtocolLoggingLevel verbose] , and opening the smtpsend log file, I can see many TLS failures:

016-07-19T12:13:14.863Z,Outbound to Office 365,08D3AFC581A92DD3,3,,,>,EHLO,
2016-07-19T12:13:14.910Z,Outbound to Office 365,08D3AFC581A92DD4,2,,,<,”220 Microsoft ESMTP MAIL Service ready at Tue, 19 Jul 2016 12:13:14 +0000″,
2016-07-19T12:13:14.910Z,Outbound to Office 365,08D3AFC581A92DD4,3,,,>,EHLO,
2016-07-19T12:13:15.004Z,Outbound to Office 365,08D3AFC581A92DD3,4,,,<,250 Hello [] SIZE 157286400 PIPELINING DSN ENHANCEDSTATUSCODES STARTTLS 8BITMIME BINARYMIME CHUNKING,
2016-07-19T12:13:15.004Z,Outbound to Office 365,08D3AFC581A92DD3,5,,,>,STARTTLS,
2016-07-19T12:13:15.051Z,Outbound to Office 365,08D3AFC581A92DD4,4,,,<,250 Hello [] SIZE 157286400 PIPELINING DSN ENHANCEDSTATUSCODES STARTTLS 8BITMIME BINARYMIME CHUNKING,
2016-07-19T12:13:15.051Z,Outbound to Office 365,08D3AFC581A92DD4,5,,,>,STARTTLS,
2016-07-19T12:13:15.145Z,Outbound to Office 365,08D3AFC581A92DD3,6,,,<,220 2.0.0 SMTP server ready,
2016-07-19T12:13:15.145Z,Outbound to Office 365,08D3AFC581A92DD3,7,,,*,” CN=*, OU=IT, O=contoso International (L.L.C), L=Dubai, S=Dubai, C=AE CN=thawte SHA256 SSL CA, O=””thawte, Inc.””, C=US 0D92CFF6070B73AD5722EC8B4DA3389B AAA3D3DADA6891A2CCB3134D0B2D7764F1351BC4 *”,Sending certificate Certificate subject Certificate issuer name Certificate serial number Certificate thumbprint Certificate subject alternate names
2016-07-19T12:13:15.145Z,Outbound to Office 365,08D3AFC581A92DD3,8,,,*,,TLS negotiation failed with error UnknownCredentials

I am sure the certificate is fine as the other hybrid servers are using the same certificate and they are able to send emails to Office 365. Also on the event viewer, I am seeing the following error:

TLS Error Office 365 Exchange Hybrid


So finally, I tried something and it worked. I opened the certificate store, and I was checking the permissions on my certificate private key, the certificate I am using for the TLS connection.

TLS Error Office 365 Exchange Hybrid2

I can see the following permissions on the private key:

TLS Error Office 365 Exchange Hybrid3


So I added the Network Service and I gave it READ access. After that everything worked just fine. Try to give EVERYONE Read access if things are not working yet.

Hope this will help someone, leave a note if it did 🙂

Office 365 and Group Moderation Tips

In the process of testing out Office 365 and Exchange hybrid configuration, an interesting thing happened that I want to share with you.

I have an on-premise Exchange 2010 implementation and couple of users are hosted at Office 365. All hybrid configurations are set and connectors are configured to route emails between the two spaces.

Everything is working fine, and mailboxes hosted on Office 365 are working just fine. Things started to get interesting when people start to send emails to moderated distribution groups.

When someone sends email to a moderated groups, and the moderator is hosted on Office 365, the buttons for Approve and Reject are not showing at his email client.

It turned out that a setting called TNEF (Transport Neutral Encapsulation Format) is causing this to happen. We need to make sure TNEF format is enabled when sending emails out to Office 365 tenant.

The TNEF setting is configurable per remote domain (Get-RemoteDomain) and (Set-RemoteDomain).

By default, there is a default RemoteDomain configured in your Exchange environment called (Default). If you hit (Get-RemoteDomain), you will see all settings that controls the behavior of email communications and format when sending emails to external parties. One of the settings is TNEFEnabled.

Now that we have Office 365 hybrid setup, the HCW creates for us a remote domain in the on-premise organization to allow TNEF (

That is great. So all what we need to do is to configure that remote domain (Set-RemoteDomain -TNEFEnabled ….) and all is done, right?

There is a small thing left to say. When Office 365 sends emails regarding moderated groups, the messages come from a system mailbox in the tenant with email address SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c} So we need to add a new remote domain called

So let us start typing some PowerShell commands:

New-RemoteDomain -Name “Hybrid Domain –” -DomainName

Now we have two remote domains:


We have then to configure both remote domains to allow TNEF format. I also recommend configuring many other settings on the way.

Set-RemoteDomain -Name “Hybrid*” -IsInternal $true -TargetDeliveryDomain $true -AllowedOOFType InternalLegacy -MeetingForwardNotificationEnabled $true -TrustedMailOutboundEnabled $true -TrustedMailInboundEnabled $true -UseSimpleDisplayName $true -TNEFEnabled $true

That’s great. Now we have configured both remote domains to enable TNEF format. I have also noticed that when on premise mailboxes try to communicate with the Office 365 system mailbox for moderation actions (Approve,Reject), they are receiving authentication errors. To fix that, add the to the address space of the Office 365 connector:

Set-SendConnector “Outbound to Office 365″ -AddressSpaces @{Add=””}

Finally, it makes sense to instruct the Office 365 tenant to treat the on premise Exchange organization the same way. Suppose my on premise domain domain is, then connect to your office 365 Exchange PowerShell, and type:

New-RemoteDomain -Name “Hybrid Domain –” -DomainName

Set-RemoteDomain -Name “Hybrid*” -IsInternal $true -TargetDeliveryDomain $true -AllowedOOFType InternalLegacy -MeetingForwardNotificationEnabled $true -TrustedMailOutboundEnabled $true -TrustedMailInboundEnabled $true -UseSimpleDisplayName $true -TNEFEnabled $true.

Reference Link 

Exchange Email Moderation Super Cool Script – Must Have

Hi everyone,

Email Moderation is one of my best features in mail flow restrictions in Exchange. You can assign one or more moderators to groups, so that if any one of them approves the email being sent, then this will release the email to that moderated group.

You can view email moderation information from Exchange GUI admin tools, but for dynamic moderated groups, you shall use PowerShell to view and configure email moderation. Usually, dynamic groups that has country or office filter criteria will contain lots of people and you want them to be moderated.

No Dashboard for moderation info

The first issue people have with email moderation is how to get a report with all email moderated groups and their moderators, and bypass moderation recipients. There is no dashboard that shows all this information in one place.

Disabled or Orphan Moderators

The second issue that Email Administrators will face is moderation list maintenance. Suppose that GroupA has one moderator called John. John decided to leave the company, and his account is now disabled. Now GroupA has no moderators. It has Moderation status set to true, but no moderators. Some cleanup job need to be performed frequently to check for the health and existence of the moderators.

Single Moderator Issue

Moreover, the best recommendation is to have at least two moderators for each moderated group, so that if one of them is not available or on a leave, the other one can moderate that group. You may want to have a regular checks to detect moderated groups with one moderators only.


I have created a PowerShell Script that you can run, and it will do the following:

  1. Generate CSV file that lists all moderated groups in your environment with the following Info:
    1. Group Name
    2. Dynamic mailing group or not
    3. Moderators list
    4. Bypass moderation list
    5. Managed By list.
    6. Email Address
    7. Alert column if a single moderator is detected.
    8. Health Field to indicate if one of the moderators is disabled or does not have mailbox anymore.
    9. Empty Moderator List warning
  2. Three Log Files will be generated. One for information, one for empty moderator groups, and one listing groups with disabled mailbox moderators.


Download the Script here

You can download the script from here Get-CorpModerationInfo

Exchange Online EOP and Send of Behalf

Some one asked me recently about an interesting scenario on which emails are send on behalf of another party, and how Exchange Online Protection (EOP) will act in this case.

There are two FROM values in the SMTP world:

  • RFC 5321  (MailFrom)
  • RFC 5322 (From)

Outlook displays the RDC 5322 From address to end users, and this is the address that is used in the user’s safe sender list.

EOP inspects both values for blocked and allowed senders and domains. Exchange Online Protection EOP and outlook handle safe sender lists differently.

In most cases, those two values are the same which is normal. Things become interesting when someone is sending emails on behalf of another party. Let us take a simple example:

Mailfrom vs MAIL RFC

  • Contoso corporation is trying to send email to their customers and they contracted with third party to send their news emails.
  • The contractor company sends the news email on behalf of contoso.
  • The email that was sent has the following values:
    • RFC 5321 MailFrom:
    • RFC 5322 From:
  • One of the customers who are using EOP receives the news email, and in Outlook, he can see that the sender is
  • The user added this address to the safe list senders.
  • Because EOP inspect both RFC from addresses, the next time an email was sent by the contractor, EOP will white-list that email respecting the user’s safe list.

Usually the RFC5321 address is the one used by EOP to do SPF checks and send NDR or bounced messages.

Reference article:

Exchange 2013 Certificate Revocation Failed

Hi everyone,

I want to share with you my personal experience in troubleshooting an interesting problem where Exchange 2013 management interface shows the status of a certificate that I had imported as (Revocation Status Failed).

So why this is happening? When Exchange 2013 tries to enumerate certificates on the computer store for you in the Exchange Admin Center, it will try to check the revocation status for each certificate to make sure the certificate is Valid. To do that, it will try to download the CRL (Certificate Revocation List) file from the internet by looking at the certificate  (CRL Distribution Points) attribute of that certificate.

CRL Certificate Exch2013

This CRL file download is happening in the background when the server is restarted and using the SYSTEM account. So the SYSTEM account is trying to download something from the internet in the background, and for sure it will use the proxy settings in the IE that is configured for SYSTEM account, which is auto detect proxy settings.

Since the server is not configured to use DHCP, then the auto discover process will go to DNS and search for ,  for example (, and since I have such record in my DNS pointing to my proxy, then the SYSTEM account is trying to connect to my proxy, perhaps authenticate and then tries to download the CRL file.

This means also that each time the SYSTEM account in the Exchange 2013 needs to connect to internet, it will do that via my proxy which is something I do not like. I would rather like to have a direct connection from Exchange 2013 to the internet, especially if we are talking about hybrid configuration and Office 365.

How to solve this issue?

I started to think, if i could log on to the computer using SYSTEM account, open the IE and remove the Auto-detect proxy setting, then the problem would be solved and i will have a direct internet connectivity that will eliminate any complexity or authentication requirements on my proxy.

So i went to one of my favorite sites [Windows SysInternals] ,  and i have downloaded the PsExec tool, and copied it to the C:\ drive of my Exchange server. This tool has the option to initiate an executable remotely or locally using local system account.

The idea is that I want to run CMD using SYSTEM account interactively and then open IE from there. Once IE is opened in front of me using SYSTEM account, i can then remove the proxy auto-detect chec kbox from there. To do that, I logged on as a local administrator to one of my Exchange 2013 where i have PsExec copied on the C drive, and then I run:

psexec -i -d -s cmd

CRL Certificate Exch2013 2

This will open a new CMD window for me. From that window, I can type WhoAmI and I can see that the CMD window is running under the SYSTEM account.

CRL Certificate Exch2013 3

Now, I will open IE using SYSTEM context.

CRL Certificate Exch2013 4

and from there I will remove the auto-detect proxy settings, so that SYSTEM will not use proxy when connecting to the internet to fetch the CRL of my certificate.

CRL Certificate Exch2013 5

Lync and Exchange Web Service Integration When Using Different Domain [Updated March 2017]

If you are have Microsoft Exchange and Microsoft Lync, then you may find this post interesting. It is about the Lync integration with Exchange Web Services EWS.

Company A:

  • AD Domain : CONTOSO.COM
  • Exchange with SMTP domain : CONTOSO.COM
  • Lync with SIP domain : CONTOSO.COM
  • Split DNS configuration.

Company A acquired a small company and they migrate them fully to their domain. Nevertheless, a couple of people wanted to have as their primary SMTP address for business need.

Now people with as their primary SMTP address, are experiencing strange and broken behavior between their Lync 2013 client, and Exchange web services. People with as their primary SMTP address, still using CONTOSO\username logons, and CONTOSO.COM as their SIP domain.



Adding TrustModelData Registry Key with value ( to the machines with Lync 2013 client that are experiencing the problem.

The registry key can be applied on a machine level or user level (See the TechNet article):

  • “HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Office\15.0\Lync\TrustModelData” “HKEY_CURRENT_USER\SOFTWARE\Policies\Microsoft\Office\15.0\Lync\TrustModelData”

There is a group policy to configure this also in the admx/adml files for Office 2013 .  This group policy setting called (Trusted Domain List) and it is mentioned here in this TechNet Article.

Exchange 2013 Capacity Planning

If you are migrating or planing to deploy Exchange 2013, you may want to know about what are the small changes that affect your capacity planning. This post will help you quickly get an overview about those changes, in order to do a more accurate capacity planing.

When you size Exchange 2013, you are sizing for three roles:

  • Mailbox Role : consolidates most Exchange components.
  • CAS Role : is an effective stateless proxy.
  • Active Directory : Exchange depends heavily in AD and you should have enough hardware for AD to support this heavy dependency.

Point 1: More free disk space on the install drive

Exchange 2013 contains built-in performance motioning components: Exchange Diagnostics Services (EDS) that collects performance data to be used by Microsoft engineers in case you open a support case, capacity planning, sizing guidance, or for performance bug detection. This is enabled by default.

There are also many other logs that are enabled by default. All of this requires lots of space on the install drive of Exchange 2013 server. A minimum of 30 GB Free space is required on the install drive.

Point 2

Capacity planning process for Exchange 2013 can be divided to three stages:

  • Read the capacity guidance.
  • Collect User Profiles and Average Message Size
  • Define constraints based on requirements
  • Input profile data and design constraints to the calculator
  • Review the output and consider the impact of each option
  • Finalize the design and do documentation.

Exchange 2013 capacity planning



Exchange 2013 capacity planning 2


Point 3 : Exchange 2013 targets balance use of hardware

  •  Rather than having set of roles in the product that use hardware in different ways and that not necessary use all the hardware on the server in the best way possible, instead now we have smaller number of roles that ideally use all the hardware that is available to that role in a balanced way.
  • Roles are loosely coupled and scale independently.

The whole idea here is that the Mailbox Role now consolidates most Exchange components, and Microsoft is pushing towards consolidating CAS Role with Mailbox Role ( also called Building Block Architecture). In this way, all the hardware available to the server will be used more efficiently.

Point 4: Memory requirements have increased in Exchange 2013

Exchange 2013 capacity planning 3


Point 5: New Mailbox Role

Most of Exchange 2013 components are now hosted in the new Mailbox role. Microsoft  also recommends to collocate the CAS role with the Mailbox role to utilize hardware resources even better.

The new Mailbox role provides simplified deployment and connectivity model. Less roles to provision and worry about, and less network packets between the old multi roles Exchange servers.

The new Mailbox role also provides  balance resource utilization and hardware efficiency, because instead of spreading roles between different servers, roles are now consolidated.

Cache effectiveness:  In previous Exchange architectures, processing and email traffic for a particular user could occure on many servers through the topology, so if i am opening my email from outlook and ActiveSync, then i may have two CAS servers processing my request. So my cached data stored on the servers would become useless as soon as those connections moved to other servers. In Exchange 2013, all workload processing for a given user occurs on the Mailbox server hosting the active copy of that user’s mailbox. Therefore, cache utilization is much more effective.

Point 6: New CAS Role

The new CAS role is now completely stateless proxy from a user perspective, so it becomes very easy to scale up and down as demands change by simply adding or removing servers from the topology. Compared to the CAS role in prior releases, hardware utilization is dramatically reduced meaning that fewer CAS role machines will be required.

Point 6: Storage Capacity

You always size

  1. Mailboxes
  2. Logs
  3. Indexes

In previous Exchange architectures, we always add 20% from the database size, as database overhead. In Exchange 2013, this overhead is now 0%

On the other hand, Content Index (CI) size is now 20% of the EDB + space for additional index set per volume (use for master merge maintenance process).

So if you have multiple databases per volume, you will have only one additional index log set. So by having multiple DBs per volume, you will save space that would other wise be consumed by the master merge log set. The master merge indexes are computed as 20% of one of the databases on the volume.

In Summary, you need to calculate index space as the following:

  1. Content index = 20% from the database size.
  2. Master Merge logs : one set per volume =  20% of the average size of the databases on that volume.

 Point 7 : Background Database Maintenance

Storage bandwidth is about things crossing between the server and the storage system. Bottlenecks were caused by the Background Database Maintenance (BDM), and in Exchange 2013, BDM is now consuming 1MB/sec/DB copy, significant reduction from 2010.

Point 7: Unified Messaging

Voice mail transaction is a heavy consumer of CPU and now the UM is part of the mailbox role now. If a server is CPU starved, then the voice mail transcription may be skipped while the voice mail is delivered.

Import from PST to online archive ? You need PowerShell Statistics Report



Well, say you have a project to import PST files to online archives in your Exchange environment. You start importing PST files to online archives one by one, and you need a way to monitor the progress and see which servers are doing the move and how long will it take to finish the current import operations. At the end, you will receive a nicely formatted HTML table to your email address.




Items Reported

This script will gather information about all mailbox import operations that are in progress (status = InProgress) and will report the following:

– User Name
– User SamAccount Name
– Office
– Targeet Database
– Percent Complete
– Queued Time Stamp
– Start Time Stamp
– Last Update Time Stamp
– Overall Duration
– Bytes Transferred
– Target Root Folder
– CAS Server doing the import process


Download the script now

You can download the script here : Get-CorpMailboxImport


Generate the HTML report with SMTP Email option
\Get-CorpMailboxImport.ps1   -MailFrom   -MailTo   -MailServer

Exchange Multi-Mailbox Search – Segregation of duties


The security or legal team needs access to search corporate mailboxes for keywords in order to investigate a security or legal incident.

Giving that person the ability to view and access other mailboxes without proper auditing is something most organization fear to do, even if that person is trusted and is a senior person.

Microsoft Exchange platform starting from Exchange 2010 I guess, comes with a new feature called Multi Mailbox Search . The problem with giving a person the ability to do searching on corporate mailboxes is still the same.

How Multi Mailbox Search works

I will not go through the details of how this feature works, as you can read on TechNet about it. Instead I will highlight couple of points:

Exchange 2010 introduces the Discovery Management Role and the Discovery Search Mailbox.  By default no users are members of this role and the user associated with the Discovery Search Mailbox is disabled and it cannot receive e-mail.

  • You start by granting a domain user “John” the role of Discovery Management in Exchange by running:

Add-RoleGroupMember -Identity “Discovery Management” -Member John

  • Then John can go to his Outlook Web App > Exchange Control Panel, and he will have access to the Reporting section under My Organization

Multi Mailbox Search

  • From there John can specify a search criteria as shown below.

Multi Mailbox Search 2

  • The results of the search will be sent to the built in system mailbox called (Discovery Search Mailbox).

John is granted automatically access to that (Discovery Search Mailbox) where he can view the results. This is because the (Discovery Search Mailbox) is configured by default with (contoso\Discovery Management) group having Full Mailbox Access. John is added automatically to that group once he is granted the “Discovery Management” Exchange Role previously.

Note: The problem with this approach is that John can perform any search or mailbox discovery on corporate mailboxes without proper control or auditing and this is extremely something to worry about.


The solution is simply a segregation of duties, where one person performs the search and other person gets access to view the result.

In this scenario, John can only go to his OWA experience and perform the multi-mailbox search with any criteria he wants, and the results will be sent to the (Discovery Search Mailbox). John should not have access to that system mailbox, and thus cannot view the results of his own search.

Now, Sue is another security administrator and she is granted full mailbox access to the (Discovery Search Mailbox) and she can see the result of the multi-mailbox search performed by John. This means that one person can do the search and cannot view the results, where the other person can view the results but cannot do the search. In other words, we require two different people to act in order to do such multi-mailbox search on corporate mailboxes.

How to do it:

  • For John, we will add him to the “Discovery Management” Exchange Role

Add-RoleGroupMember -Identity “Discovery Management” -Member John

  • For Sue, go to Exchange Management Console, search for “Discovery Search Mailbox”, right click and click “Manage Full Access Permission” and do the following:
    • Remove CONTOSO\Discovery Management
    • Add CONTOSO\Sue

Multi Mailbox Search 3

  • Ask John to do the multi-mailbox search from his OWA experience
  • Once done, the results are sent to the “Discovery Search Mailbox”, and John cannot view it although he is member of the (Discovery Management) role, but he cannot access it as we removed the full mailbox access from that mailbox for the AD security group “Discovery Management”.
  • Now John will call Sue and asks her to access that discovery mailbox by typing:

Note: you can get the discovery mailbox SMTP. You can figure out this SMTP by searching for the “Discovery Search Mailbox” in the Exchange Management Console and view the SMTP address from there.

Multi Mailbox Search 5