Windows 7 slow logon troubleshooting

Hello,

In this article we will discuss how to troubleshoot Windows slow logon using Windows Performance Recorder (WPR) and Windows Performance Analyzer (WPA).

The case:

Windows logon takes about 1 minute to display the desktop, on a DELL Latitude Laptop, with a I7 CPU and SSD hard disk.

To deal with this issue, I first started Windows Performance Recorder to create boot trace.

You can find in this article how to install and use Windows Performance Recorder https://zinetek.wordpress.com/2015/12/16/how-to-use-wpr-to-record-boot-sequence/

Analyzing the trace

After the system reboot, and the trace created I open it in Windows Performance Analyzer (WPA).

First, lets take a look to the system configuration. Go to tab trace, then system and then General.

WPA_sysconfig

Next click storage

WPA_Sysconfig_storage

As we can see, this Laptop is an I7 3Ghz CPU, 8 Gb of RAM, and SSD hard drive. it becomes obvious that with these specs, 1 min is too long to logon.

So now lets go deeper in our analyzis. first, Add Boot phases graph in the analysis Window.

wpa_boot_phase

We can see that the major delay in the boot trace comes from the winlogon phase (59 s).

Many operations can occur during the WinlogonInit phase; PnP services, Network subsystem, Group policies processing, credentials input can all lead to a delay.

Before we dig in the analysis, lets take a look to the computation and storage graph to find out where to look first. Disk and CPU performance are the must popular causes of slow logon.

According to the two graphics, there is no issue in the cpu or disk usage. So we have to look elsewhere.

WPA_Computation

WPA_Storage

Now we are going to add the Generic events graphic and expand the “Microsoft-Windows-Winlogon”

Genericevnt_winlogon

The “DisplayWelcomScreen” aka CTRL+ALT+DEL was available at 6,84 sec of the trace and the user entered the combination in the keyboard at 7,46 sec of the trace (less than 1 sec for the whole operation).

Next in the “RequestCredentials” task, the user entered his username and password in 4,22 sec (11,68 – 7,46).

all others tasks doesn’t took a lot of time except the “RestoringNetCoonections” that took more than 66 sec. I think we caught our culprit.

So the OS tries to reconnect the network drives during the winlogon process.

After the desktop shows up, I opened the windows explorer and we can see for network drives with a red cross which means that windows was unable to reconnect them. Those drives are mapped on a remote shares.So when the VPN connection is disturbed due to a bad Internet connection, then user experience this dekay when logging in.

Lecteurs réseau 02_modif

To be certain that this is the root cause of our logon’s delay, I disconnected all the network drives and made a new boot trace and here is the result:

WPA_Winlogon_afterresolution

Now that the root cause of the delay was found, the question was, how to make the network drives available to the user without causing major delay in the winlogon process?

The answer I came up with, was to create a script that runs after the user has opened his session.

 

How to use WPR to record Windows boot process

 

In this article I will show you how to use “Windows Performance Toolkit” to make a trace of Windows boot sequence, in order to troubleshoot slow logon.

First of all, you need to download the Software Development Kit (DSK) https://dev.windows.com/en-us/downloads/windows-10-sdk

After running the sdksetup.exe you should select one of the following options:

SDK setup

The first option will install Windows performance tool kit on the computer running the setup. The second one, will allow you to download an offline setup files that can be executed on an other computer.

For our purpose we will chose the first one.

Click next and accept the license agreement.

SDK setup 01

Click on “Windows Performance Toolkit” and then install.

Reboot your computer to finish the setup.

To make a trace of your boot sequence, type “wpr” from the windows start menu and then click on “Windows Performance Recorder”.

run_wpr

wpr01

On the “Performance scenario” menu choose “Boot”.

wpr02
Type “1” for the numbers of iterations and then click the sart button.

wpr03

Select the path where the trace file (.etl) will be saved and click on the “Save” button.

wpr04

After you click on the OK button your system will reboot and “Windows Performance Recorder” will record all the boot phase.

After you open your windows session WPR will end the trace and will save the file in the specified path.

Boot_trace_inprogress

Generaly the trace file will be a hundred of Mb till Giga bytes.

So if you want to share your trace or send it by e-mail, don’t forget to compress it.

DELL Inspiron 3543 freezes

The issue:

We bought a new DELL Inspiron 3543  Laptop for one of our users. By doing the nessery configurations (install AV, Office, Join to domain) Windows Freezes after a while. So we do restart the computer and after a while the same thing…Windows freezes !!
It appears for many times and at différent state of usage (installing software, running programs, or doing nothing).

So why a new PC Freezes like that?

We returned it back to the supplier and he said there is nothing bad everything is fine. And sure enough when we get the Laptop back the problem arises again!

The solution:

So the next time it did that, I took the dump generated by the os and located under c:\windows\mindump. Analyzing the dump with windbg utility by typing the !analyze -v command to display informations about the current exception.

The result shows that the exception occures in the iaStorA.sys file.

Windbg Modif
So what is the iaStorA.sys file?

A quick search on the Internet shows that is a Intel SATA Controller driver. This file is located in c:\windows\system32\Drivers and the file’s version seems to be an old version.

Sans titre (2)

So I went to the Dell Webpage and search for a recent version of Intel SATA controller and I found one of june 2015.

So by downloading and installing this new driver’s version, the problem was solved.

identifying suspicious network activity by using Wireshark and Process monitor

Intoduction:

In this post I’ll discuss on how to identify suspicious network traffic using Wireshark and Process monitor.

The case:

I’ve started the packet capture on my PC with WireShark and one thing captured my attention. I’ve seen many packets intended to a pc that doesn’t exist any more in the network sent by the file server!

NBNS_query_boulwa_Modif

So why the file server with the ip adress 10.x.x.6 is sending NBNS queries (NetBios Name Service) to the host BOULWA-XP asking for his IP adress?

WireShark can show me packets sent from the file server to the specific host, but it can’t tell me which program or service running in the file server that is responsible for this trafic.

To find this program or service I’ve used Process Monitor from SysInternals tool. So I started the capture for a few seconds, then I did a search on the string “BOULWA-XP”. In the result we can see the process name at the origine of the query, in this case it’s spoolsv.exe. Next I’ve applied a filter to have only the traces related to spoolsv.exe

procmon_search_boulwa_modif

spoolsv_filter

spoolsv_filter_01_modif

In the filtred trace, we can see also the spoolsv.exe process accessing the “HKCU\Printers\Connections\,,BOULWA-XP,Microsoft XPS Document Writer” registry key. This means that there is a connection to the printer “Microsoft XPS Document Writer” on the host BOULWA-XP. It can be verified by opening printers location in the control pannel.

panneau_cfg_impr_modif

So by deleting this printer from the control pannel, the network traffic related on this printer disapears from the network.

Cannot scan to shared folder using canon IR copier

The case:

We have a corporate Canon IR copier with scan over network feature enabled. But we cannot scan to the shared folder at the smb server.

Carnet_adresses_modif

So, lets find why.

At the beginning I suspected a permission issue on the share, so I checked the permissions that seems correct, everyone is given read and write acces to the share.

Second, I started process monitor on the computer who hosts the share, but it did not give me any clue. So I deduced that the issue is related to the network and I have to do a captue the packets on the Canon’s Ethernet interface.

To do so, I set up the SPAN (Switch Port Analyzer) feature on the Cisco Catalyst Switch to copy all the network trafic from the Canon’s interface to another interface on which I plugged a PC with Wireshark installed on it.

Here is the wireshark’s capture result:

Wireshark_netbios_issue_modif

We can see the canon copier sending Netbios broadcasts asking for the IP address of the host named AL1XXXX7P (The smb server). As it didn’t receive any response he sends a dns query to the DNS Server asking for the IP of the AL1XXXX7P.DOMAIN.LOCAL host, and the DNS server replies with the IP 10.x.x.13.
Next the copier sends a Netbios query to the 10.x.x.13 host trying to initiate a communication on UDP port 137. the smb server replies that he can’t communicate on port 137.

Ok, now we know why the scanned documents are not sent to the the smb share. So Why the SMB server cannot use his UDP 137 port?

The reason is that, a few days ago, I’ve enabled the “Disable Netbios over TCP/IP” option of the network interface. By enabling Netbios over TCP/IP again I was able to send scans again from the Canon copier to the SMB Server.

NIC_Netbios_disabled

In the following screenshot the Wireshark trace when the the Netbios communication betwen the two devices succeed:

Wireshark_netbios_success_modif

Case of unexplained LDAP requests

In this post I will explaine how I troubleshooted an unusual LDAP requests on the network.

The case:

A computer member of windows domain is sending LDAP requests massively and constantly on the network, to all the domain controllers of all sites.

The computer is on site A (Subunet 192.168.1.0/24) and sends UDP packets on port 389 to the domain controllers located on the following sites:

– B (subnet: 192.168.2.0/24)

– C (Subnet: 192.168.3.0/24)

– D (Subnet: 192.168.4.0/24)

The requests appears on the entreprise Firewall, and the IT manager of one of the remote sites, starts complaining about this unusual udp packets sends to his domain controllers. So I decided to find out what is causing these massive LDAP requests.

Kerio_ldap_modif

The investigation:

In the firewall, I’ve terminated all the UDP requests on port 389 (LDAP) initiated by the host 192.168.1.89. I started Process Monitor (procmon) and waited untill the host starts sending UDP packets to the domain controllers. After that I stopped the procmon’s capture and started my investigation.

The big question was: which process is sending this LDAP requests? And why is it doing it?

So in process monitor, I set a filter on the Path column that contain 389 value.

Filter_path_389

The result of applying the filter is the following:

NB: for confidentiality reasons I hid the original IP addresses and assume that subnets are 192.168.x.x

Filter_path_389_result1

We can notice that the “Lsass.exe” process is sending UDP requests on port 389 to the local and remote domaine controllers.

What is lsass.exe process in fact? Lsass.exe for Local Security Authority Sbsystem is a process in Microsoft Windows operating systems that is responsible for enforcing the security policy on the system. It verifies users logging on to a Windows computer or server, handles password changes, and creates access tokens. It also writes to the Windows Security Log (https://en.wikipedia.org/wiki/Local_Security_Authority_Subsystem_Service).

So now we have to find which process is calling lsass.exe to authenticate on the domain controllers?

The answer is found upon above in the trace.

lsaas_read_sophos_regkey

The lsass.exe process is accessing a registry key “SophosSAUALGUCT120000” which is in fact the concatenation of “SophosSAU” for Sophos Antivirus Update program and the netbios name of the Windows PC.

Having this clue between hands, I stopped the Sophos Update service from service manger and observe if there is any LDAP request sent. And there was no LDAP query sent for a while. But when I start the Sophos Update Service the LDAP reapprars quickly.

The culprit was found, but what was causing Sophos Update Service sending this unusual LDAP requests? After having a deep look to the sophos installation in the PC it revealed the the installation was damaged. After reinstalling the Sophos antivirus again the problem was solved.

Conclusion:

Without tools like Process monitor, it would be difficult to troubleshoot this kind of issues.

You can download pocess monitor by clicking: https://technet.microsoft.com/en-us/sysinternals/bb842062.aspx

Troubleshoot access issue to a shared folder using Sysinternals tools

Introduction:

In this post I will show how to troubelshoot an access issue to a shared folder using Sysinternals tools.

The case:

In Symantec Backup Exec, I wanted to add a new storage device as a share. The shared folder was created in a Synology NAS, with domaine administrator rights including my username. But I got the following error: “The path seems to be invalide. Check the server and the paths names”.

Chemain inaccessible_modif

So I tried to access the shared folder with Windows Explorer…and it works!

Nas_Share_From_Explorer

So why Symantec BE is unable to access this share?

To discover it, I run Process Monitor and I take a trace when I clic on the Next button.

In process monitor I do a search on the “\\NASSYNOLOGY\Backup” string and the first result give me a clue. The process “BackupExecManagementService.exe” trying to access the “\\NASSYNOLOGY\Backupp” share is getting “ACCESS DENIED” as a result.

Procmon_access_denied_modif

The next step is to determine which user is running the BackupExecManagementService.exe process. To do so, I use the SysInternals’ Process Explorer tool.

In process Explorer, right clic on BackupExecManagementService.exe, then clic on properties and then on Security tab to get the user name who ran this process.

ProcExp_BE_access_user_modif

Effectively this user were not granted the access rights on the shared folder. By giving it the correct access rights the problem was solved.

You can dowload the Sysinternals suite here: https://technet.microsoft.com/en-us/sysinternals/bb842062.aspx