EM Server 5.8.2 : AD Authentication

Hi team,

We are doing our first tests with EM Server 5.8.2. We only use Active Directory Authentication.
In the afternoon I tested the authentication with someone in parallel. I have privileges, he has not.

Something weird happened. On my side everything was good : my login was displayed at the upper right corner. But on my colleague's side, it was strange => he had my access and the login displayed was mine, although he should had no access at all.

I can not replicate it. We were doing the test at the same time. I think you may find in the code something wrong. It must be about the session management maybe. I see no reason why he would be able to log as myself. It was really a big security issue.

Edit : I'm interested to know what is happening when we click on "keep me signed in". This can be related to this option.

Hi
We will definitely review all possible scenarios.

Could you please provide more details about how the simultaneous login was carried out?
For example, did you use two different computers? Were there different AD user accounts on those computers?
Did you or a colleague enter the username and password for an AD account?

Also, please check the user.session.started events in the journal database for the relevant time period. The initiator_detail field will show the AD user account name, and initiator_ip_addr will show the client’s IP address.

If that option is enabled, the active session remains valid even after the server restarts.

UPD-1

Could you explain in more detail why you said “although he should have had no access at all”?
For example, did he not have an EasyMorph Server account at all? Was his account blocked? Or did he have an account but no assigned roles? Was his account created automatically when he logged into the EasyMorph Server?

It must be about the session management maybe.

Let me clarify: on the server side, the session token is stored as a hashed value, while the client still sends the unencrypted/unhashed (original) session token. Whenever the server needs to identify the user, it hashes the token that the client sends and then searches for that hash. Because the server never stores the original token value, it’s impossible for the server to assign the same session to two different clients.

I will do more tests with the guy.

He was trying easymorph server for the first time and he was not declared at all on the server : no account, no role. He is using a different computer, with different IP and different AD login, nothing related with my account/computer.

So on one side I was connected with all my privileges, and on the other side he was supposed to have no access at all, the perfect unknown. And he showed his screen to me : my login was displayed. On our side we are going to check if it comes from our architecture.

Edit : it's replicated with another user, completely unknow by the server also. He could see my login. Note that I use the "keep me signed in"' option. Also the user could see very quickly a red banner at the top of the screen but it immediately disappeared. We could not see if there was a message. Note also that the bug was short : after one minute my colleague could not replicate the issue.

Edit2 : If I remember well, the unknown user had not my privileges, it was only the display of the login that was false. So I guess the problem is on the display of the login in the upper right corner. The sessions may be well managed as you said earlier.

Edit3 : I was also able to log and see in the upper right corner the login of my colleague, so the extact opposite situation. I really think there's a bug in the display of the current connected guy. But my privileges are applied. It's just a display bug.

We are looking into the code and reviewing different scenarios to figure out why you are experiencing this issue.

it's replicated with another user, completely unknow by the server also. He could see my login.

Please note that in the top-right corner, the server can display either the userLogin or the Name (if it is set). The Name takes precedence over the userLogin. You can change both Name and userLogin in the Users section. Check that the correct values are set in the Users section.

Note also that the bug was short : after one minute my colleague could not replicate the issue.

Could you please tell us more about what changed during that minute?
Did the displayed name change to the correct value on its own?
Did the user close the session?

I was also able to log and see in the upper right corner the login of my colleague, so the extact opposite situation. I really think there's a bug in the display of the current connected guy...But my privileges are applied

Could you please clarify whether this issue occurred right after logging in, or if it happened at some point later on during your session?
How long was the incorrect name on display?
Also, how have you confirmed that the permissions were set correctly during this time?

Also the user could see very quickly a red banner at the top of the screen but it immediately disappeared. We could not see if there was a message.

This can happen if the user is already logged in.
Please double-check that your browser profiles are not syncing across multiple devices or users, as that could cause this unexpected behavior.

Only the AD login appears, just as in 5.7, thats the expected behaviour

Nothing changed during the minute

We only tested authentication, nothing more : AD directory > sign in. Thats the only test. To replicate with a new user the scenario was :

  • I sign in with keep signed in
  • Another user (whoever) connects (no checkbox)
  • The first time this user could see his login
  • Logout login again from this same user (I was still connected)
  • => my login appeared on his screen

Logout / login several times after => no issue

We dont have multiple devices, only one.

Please complete the following steps:

  • Connect to the Journal database.
  • Filter the session start events (user.session.started) so that only those matching your IP address (initiator_ip_addr) are returned.
  • Review the initiator_detail field during the time period (the timestamp field) when you observed the server displaying a username that wasn’t yours.
    • Check if there are any rows in the table that list a different username during this time.

Below is a snippet of the relevant SQL query:

SELECT
   *
FROM "main"."em_server_journal_events"
WHERE "event_name" = 'user.session.started'
    AND "initiator_ip_addr" = '10.20.30.50'
ORDER BY
    "timestamp" DESC

Good thing to add :

Our architecture is special. We don't connect to the server directly. In this environment we have One Load Balancer => One web front solution => One Load Balancer => EM Server. So your query will not work because the "initiator ip" is always the Load Balancer, not our IP.

Is the display of the user connected based on the property initiator_ip_addr ? I hope not because in our case this will not work.

Is the display of the user connected based on the property initiator_ip_addr ? I hope not because in our case this will not work.

Certainly not.
I just meant that using that method would be the easiest way to see which specific Active Directory user started the session.
After all, when working with Active Directory, EMS completely depends on the information it receives from the domain infrastructure.

In this environment we have One Load Balancer => One web front solution => One Load Balancer => EM Server.

So, if I understand correctly, this same infrastructure has been in use for a while now.In other words, no changes were made that could affect the NTLM challenge or any specific traffic routing scenarios?

UPD

When NTLM is used behind multiple load balancers (or even one load balancer that’s incorrectly configured), there is a risk of “challenge/response mismatch” that could cause the wrong user session to be associated with a request.

I've just confirmed that the NTLM challenge requires sticky sessions on the load balancer. Have you already configured them?

We have experts who are used to setting up this kind of architecture with load balancers but I will ask them to double check. In our first architecture we had load balancer => EM Server which was doing very well. This architecture is different : LB => Web server => LB => EM Server. Maybe it's not compatible with Easymorph ?

Edit : Note that if it was a NTLM issue, the user who saw my login on the top right corner would have had the same privileges as I have. But it was not the case : he could only see what he was supposed to see (default workspace). That's why I think it's more related to the display of the connected user.

All of the user’s session data — including display name, effective permissions (for each space and system), and the session’s expiration time — are stored together in one session object, which is associated with a session token. So far, we have not found a case where this object is built incorrectly, but we will carry out some additional checks.

This means that if you happen to see someone else’s display name, it is very likely that you are logged in under that person’s credentials.

Please ensure that the load balancer configuration is correct. If the NTLM challenge is misconfigured, it could cause sessions to be opened under someone else’s account.

To confirm, you’re currently using EMS 5.7 in a setup of [load balancer => EM Server], and that works correctly. But when you try EMS 5.8.2 with the architecture [LB => Web server => LB => EM Server], you encounter issues when signing in via Active Directory. Is that correct?

Finally we can easily replicate the issue : 2 users connected using AD authentication must click on "Sign In" several times during the same period. If they insist a bit, one will be connected as the other and vice versa.

At the beginning I thought it was only the display of the login but no : you really are connected as the other, with the privileges of the other. For example I am the only admin and I tested with another user : he could log as myself and he had all admin privileges.

Note that our achitecture does not seem to be problematic : it 's used in a lot of other solutions AND we are not able to replicate the issue in EM Server 5.7. So we are quite sure it's related to 5.8.

Note that we could replicate without checking the checkbox "Keep me signed in". It is really problematic.

Hi.

It appears you haven't addressed the questions I asked in my last post.

To confirm, you’re currently using EMS 5.7 in a setup of [load balancer => EM Server], and that works correctly. But when you try EMS 5.8.2 with the architecture [LB => Web server => LB => EM Server], you encounter issues when signing in via Active Directory. Is that correct?

Please ensure that the load balancer configuration is correct. If the NTLM challenge is misconfigured, it could cause sessions to be opened under someone else’s account.

UPD
To put it another way, does the issue still happen when you connect directly to the server/WebUI instead of going through the load balancer?

I shall ask my prod team to bypass the web server and test again. I will communicate the results of our tests.

FYI, we have replicated the configuration of our first platform (still in 5.7) with no web server in front of EM Server. It's working fine. So I guess we can say it's not coming from 5.8. I noticed that the web server was in HTTPS / 443 but the communication between the web server and EM Server was with port 6330. Don't know if it can explain. We get back to 443 everywhere, add the web server and see if it happens again.

Ok, it seems to come from the architecture LB => Web server => LB => EM Server. When we do LB => EM Server, no problem. I understand that you have a IIS component automatically installed.
May it come from this ? I mean if we put a web server in front of EM Server, what would be the ideal configuration to make it work ? Should we stop IIS service on EM Server in this case ?

Hi

I understand that you have a IIS component automatically installed.

As far as I know, we do not install any additional IIS components and do not use or support IIS for hosting. We do, however, rely on http.sys, which IIS also uses internally.

Right now, it’s difficult to give concrete configuration advice without knowing more about how your two load balancers are set up. Perhaps if you send a more detailed technical description to support, we can identify possible points of failure.

In any case, you’ll need to ensure that traffic is being routed correctly from the client to the final server. For NTLM challenges, your load balancer should have the appropriate sticky session configuration or other options enabled that guarantee NTLM works correctly. Sticky sessions typically rely on cookies, so make sure the necessary cookies are making it through the entire pipeline.