SCOM : How to resolve “The SMTP Local Retry Queue Total is outside calculated baseline” error in SCOM.

Hi
As the notification is clearly saying that there is some error in our exchange server’s Local retry Queue. But what is the exactly happening.?
Let explore it little bit.
Error :::
Right Clink on the Error, Click on Open and Choose “Health Explorer”

Now Click on “SMTP Local Retry Queue - Queue [Exchange Queue] , and click on “Monitor Properties”

In “Performance Counter” tab, it is showing that this monitor is based on “Perfmon” and it is monitoring using the “Local Retry Queue Length” on “SMTP Server” Object of the Exchange Server.

The Next tab in “Baselining”, which indicates that this is a “STT” Self Tuning Threshold Monitor. OK,,

Now click on “Overrides” tab, and click on “Overrides”

Choose “ For All Object of Class Exchange Queue”

in Overrides the “Inner Sensitivity” of the monitor is 3.11 and I think it means if it have more then 3.X message in the Local retry queue it should send an alert.

how to check????
why not we manually check the “Local Retry Queue” performance counter manually, Isn’t it is a good idea?
let’s do it, Open the “Performance Monitor” in Object choose “SMTP Servers” and in Counter choose “Local Retry Queue Length”
in Counter Explanation is says “ The Number of messages in the local retry queue”

Ok, so how many local messages are stuck, let check, OK , we have 4 local message stuck in the retry queue.

now go to your Exchange Server Queue and you will find that there is 4 messages are stuck.

let’s find them , Delete them if they are not necessary emails.

Once the Messages are deleted, our local retry queue is back to normal i means on Zero 0.

after 5-10 minutes the error should be gone otherwise you can choose the monitor and click on “Recalculate health” Option.

YippY!!! Exchange Server is Happy Again.

Thanks
Aman Dhally

Get-Overrides created on Specific Day .

hi,

yesterday one of our  SCOM admin created few overrides in SCOM and today he is on leave and the also not picking up the phone and I need to know which overrides he created.

Then i think that lets try to write a little basic PowerShell script which show the list of overrides created between specific number of days.

you can download the script from here : http://dl.dropbox.com/u/17858935/Get_SCOM_Overrides_by_Day_Created.zip

Make sure you run this script in “Operations manager Shell”

   1: ### I set $olddate to 2 Days ago date

   2:  

   3: $olddate = (Get-Date).AddDays(-2)

   4:  

   5: ## Select every Management pack and Piped it to "Get-Override)

   6:  

   7: $Mp = Get-Managementpack | Get-Override 

   8:  

   9: ### Now it will show only overrides which are created after $oldDate

  10:  

  11: $Mp| Where-Object { $_.TimeAdded -gt $olddate} | select ManagementGroupId,Name,TimeAdded | fl *

  12:  

  13: ######## E N D of S C R I P T #############

in $olddate i minus 2 days so if today is 18 November then $olddate should have 16 November stored in variable

in variable $Mp in am storing all Management packs and piped them to Get-Overrides

lets run $Mp lets see what we will get.

It shows the list of all Overrides in all management packs. Now we need to sort them.

$Mp| Where-Object { $_.TimeAdded -gt $olddate} | select ManagementGroupId,Name,TimeAdded | fl *

in above command , we piping $MP to where-Object cmdlet and choosing TimeAdded property in Overrides and comparing them with our variable $OldDate , so if the TimeAdded is property is greater then 16 November then it show all the Overrides created between 17,18 November.

seems working … :)

Download Link: http://dl.dropbox.com/u/17858935/Get_SCOM_Overrides_by_Day_Created.zip

Hope someone like it :)

Thanks

Aman Dhally

Thanks to www.PowerWf.com

 

to mention our post on there Blog :)

http://blog.powerwf.com/post/12114067606/using-powershell-to-see-if-agent-proxy-is-enabled-in#disqus_thread

 

Thanks Guys

 

Aman Dhally

Agent Proxy on SCOM Agents

Hi,
When we install SCOM agents on any server or workstation after installation we need to enable [tick on] Agent Proxy on Agents properties via go to :

• Administration Pane

• Go to "Agent Managed" beneath "Device Management"
• Search for the Agent
• Right click on the Agent and select Properties
• and tick on "Agent Proxy"

good Enough !!! but if we  installed SCOM Agent on 100 servers or if your organization have more then two administrators and rather then you other administrator installed the SCOM Agents and he forget to enable the “Agent Proxy” , then you you found that which Agents are “Proxy Enabled” and which are not.

There are two ways to accomplish it.

Option 1:

Check the Settings of each and Every SCOM Agent

Option 2

Use PowerShell 

let me show you how .

Open “Operations Manager Shell”

and type this command “Get-agent | ? { $_.ProxyingEnabled -match $False} ” and hit enter .

and this will show you the list of All agents which doesn’t have “Agent Proxy” enabled.

to be more precise we can also select name of the Agents

“Get-agent | ? { $_.ProxyingEnabled -match $False}  | select Name”

That’s all 

now we know on which “Agent” we need to enable “agent Proxy” setting.

Thanks

Aman Dhally

SCOM: Use Operations Manager Shell to close multiple Alerts generated by Same Rule

Hi,

When today i login in to my SCOM console I saw approx 134 Active Alerts about Microsoft SQL Job failure. I was about to close these alert but then one thing strike in my mind that lets try to close these alerts using “Operations Manager Shell”. These Alerts are generating from same source and the name of the alert is same to it would not be to hard. So let try.

Our First Step is to find out the Command which can show us Alert in SCOM Shell.

Open “Operations manager shell” and type  Get-Command *alert* , this will search for all SCOM cmdlets which have the word alert (we use wildcard *), and as you know PowerShell cmdlets works on Verb-Noun format  so if we use Get-Alert cmdlets it will shows all alerts.

Let Try Get-Alert

Type get-alert in the shell and hit Enter and it shown you all alerts.

Next task is to choose which Alert to Close .. if you look at active Alert in SCOM CONSOLE name of My Alert is “"A SQL job failed to complete successfully"

 

Now our next step is to find the properties and methods supported by Get-Alert command. To know these we need to use another command Get-Member

Yes.. it has name property…Gr8

Now we need to see all alerts whose name match “"A SQL job failed to complete successfully", for filter the output from Get-alert command we pipe (|) the output of Get-Alert command to Where-Object cmdlet.

Get-alert | Where-Object { $_.Name -match "A SQL job failed to complete successfully"}

 

This will show all alerts whose name matches with "A SQL job failed to complete successfully"

So now our next step is to find a cmdlet which can close these alerts. lets find out..  run the same command

Get-Command *alert*

So this time we have find Resolve-Alert cmdlet. now we need to join and our cmdlets command using piping . so this should be like this.

get-alert | where-object { $_.Name -match "A SQL job failed to complete successfully"} | Resolve-alert

in Get Alert we are searching for an Alerts | then we are filter then using Name with match to “Sql job Failure | and then we are resolving them.  now type the above command and hit enter. It will take sometime to do this.

When you command run successfully, open SCOM Console and search for same SQL JOB error and you will found nothing :)

 

I hope that it helps someone…

Thanks

Aman Dhally

Mystery of Blank Schedule Email Reports in SCOM

Hi,

After deployment of Reporting Server for SCOM we test the working of Reporting Server by generating few “Windows Generic Reports” manually and they works perfect. So we plan to schedule Reports for a weekly delivery on Emails. So that we can have the statistics of our server weekly.

So when we test schedule email reports in SCOM, but when reports arrives in an email they are with no data completely blank email with link to Reporting Server. Then we investigate on internet and found that this is a knows issue (KB:  http://support.microsoft.com/kb/972821) and we also found  the blog posting on Kevin Holman on the same Issue. (Please visit his blog posting  for more clarifications..click here )

So Lets try.

Step:1

Login to your SQL Reporting Server for SCOM

Navigate to default Reporting Server installation path and navigate to ”Reporting Services\ReportServer\bin”   and Open

“ReportingServicesService.exe.config” in notepad.

“C:\Program Files\Microsoft SQL Server\MSRS10.MSSQLSERVER\Reporting Services\ReportServer\bin”

 

 

Step:2

After opening “ReportingServicesService.exe.config”  we need to insert below code in to this file. The main issue is where to insert the Code.

 

To resolve this problem, add the following information to the ReportingServicesService.exe.config file in the ReportServer\bin directory: <dependentAssembly> <assemblyIdentity name="Microsoft.ReportingServices.ProcessingCore" publicKeyToken="89845dcd8080cc91" culture="neutral" /> <bindingRedirect oldVersion="9.0.242.0" newVersion="10.0.0.0" /> </dependentAssembly> <dependentAssembly xmlns="urn:schemas-microsoft-com:asm.v1"> <assemblyIdentity name="Microsoft.ReportingServices.ProcessingCore" publicKeyToken="89845dcd8080cc91" culture="neutral" /> <bindingRedirect oldVersion="9.0.242.0" newVersion="10.0.0.0" /> </dependentAssembly>

The Right Position of Correct code is below.

This XML section must be added in between <runtime><assemblyBinding…> and </assemblyBinding></runtime> statements of the \ReportServer\bin\ReportingServicesService.exe.config file.

 

After Insert the Code in to “ReportingServicesService.exe.config” . save the file and reboot the “SQL Server Reporting Service”

That’s all :) ..

BUT…

But i after doing that problem is still not resolved. We still getting Black Reports. Now No Clue what to do next.  Then i tried the Following. I open the File in “Notepad++” and formatted it, save the file and restart SQL Reporting Server.

Before Formatting

After Formatting

BingO.. this works for me … ( I know it doesn’t make any sense but it works) .. After done this my all schedule email reports works perfectly.

 

I hope that it may save some one else time ..

Thanks

Aman Dhally

SCOM: Run As Account does not exist on the target system or does not have enough permissions [MS SQL server 2008]

Hi,

Yesterday  my SQL Server Management Pack reporting that it is facing some problem with “run as accounts” it says either “run as accounts does not exists” or they don't have “enough permission” to do some tasks. Both these server have SQL Server 2008 installed.

When SQL 2008 is installed by default, it no longer places BUILTIN\Administrators in the SQL security access list.  The install of SQL 2008 now prompts the installer to give SQL a user account, or Group, to grant SA (SysAdmin) rights to.  If installing on a standalone instance, NT AUTHORITY\SYSTEM (Local System) is still granted SA rights.  If installing on a clustered instance, NT AUTHORITY\SYSTEM (Local System) is granted public rights, but not SA. Kevin Holman

Error

Kevin Holman wrote an excellent post on this error.. click here

I have only few machine which are running SQL Server 2008, so i plan to give my default agent action account the rights on SA in SQL Server 2008.

Solution

Login to problematic SQL Server. Open SQL Management Studio. Got to SECURITY and then expand LOGINS,

Choose the “Default Agent Action Account” right click on it and then choose Properties.

Now click on “Server Role” and then select “SysAdmin” and click on OK..

That’s All :-) .. It should resolve the SQL Server 2008 Run As Problem.

I hope that it helps someone :)

Thanks

Aman Dhally

SCOM: Event ID “2000”

Hi,
Today I check my SCOM windows event log and saw some error logs with “EVENT ID 2000”. Complaining about that the one of my  SERVER is not a part of SCOM Management Group.


So Lets try to resolve it .
Step 1
My First Step is Login to the server who is generating an alerts. First I removed the SCOM Agent and then I reinstall   again manually.
Step 2
Now in SCOM console to to “Administration” then click on “Pending Management” and you may see the name of the server on which we just installed the SCOM AGENT.

Right click on Server and click on “Approve”


Now wait for 3-4 minutes to update the changes in SCOM and after few minute you will see that server is appeared in SCOM and it is healthy now.

And you will see “EVENT ID 29103” saying that our problematic server is now successfully able to contact with SCOM server.


I hope it may helps someone
Thanks
Aman Dhally