In this section, the following is discussed:
- What's the purpose of MessageBox Data Monitoring
- Setting up a MessageBox Data Monitor
- Archive And Terminate Suspended Service Instances
- MessageBox Data Monitoring Email Notifications
What's the purpose of MessageBox Data Monitoring
When messages flow into the BizTalk Server, the messages get persisted in BizTalk Server's MessageBox database. The service instances might move into different states such as Suspended (Resumable), Suspended (Non-Resumable), Active, Ready to Run, Dehydrated and so on.
Once the service instance moves into a particular state that demands user intervention, this forces the BizTalk server administrator to manually log in to the BizTalk Server Administration Console and fix the problem with respect to the service instance. BizTalk360 solves this problem through the Message Box (Queries), offering administrators to query the message box database for such messages and take action without having to log in to BizTalk Server Administration Console.
Despite offering this capability to BizTalk administrators, BizTalk360 goes one step further to reduce the manual intervention that is required from the administrators. Through the Message Box Data Monitoring concept, administrators can choose what needs to be done when a service instance becomes suspended. For e.g., the administrator can set up an alarm like "If there are >20 Suspended Service Instances between 09:00 AM and 05:00 PM, resume all the instances". He can simply log in to the BizTalk360 Data Monitoring Dashboard to see the status of the message box data for the day. He can also set up email notifications for the alarm. By doing so, the administrator eliminates the need to often log in to the BizTalk360 application and check for the status of the service instances.
Setting up a MessageBox Data Monitor
Follow the steps as shown below to create a MessageBox Data Monitoring alarm:
- In case you did not do it earlier, follow the steps as mentioned in this section to create a Data Monitoring alarm
- Select the 'Use this alarm for Data Monitor Alerts' checkbox to map the current alarm to Data Monitors. Only when this option is selected, the 'Notify on Success as well' checkbox gets activated.
- The 'Notify on Success as well' checkbox is optional for Data Monitoring, while the Use this alarm for Data Monitor Alerts option is mandatory.
- Click OK to save the alarm information
- Choose Alarm - As mentioned above, the first step to set up message box data monitoring is to create an alarm. Once you have created the alarm, the alarm name will be displayed in the dropdown. Select the alarm that should be used for message box data monitoring purposes
- Monitor Name - A meaningful name for the Message Box Data Monitor. Say, as per your requirement you want to terminate the suspended instances after every 15 minutes when the count goes above 20. The monitor name can be set as "Terminate suspended instances every 15 minutes when count > 20"
- Query Type - This section is very similar to the Message Box (Queries) section in BizTalk360. Select the type of query for which you want to run the data monitor. As per the above requirement, since we are looking for suspended instances, select 'Suspended Service Instances' from the drop-down
- Add a filter - To filter the query results based on different parameters (application name, host name, instance status, service class, service instance id, service name, service type id)
- Warning Threshold - Select the value from the drop-down on when you want to be notified when there is a warning. The first drop down is to select the Operator (> or =) and the second operator is to select the warning value. For e.g., When the number of service instances are > 10, throw a warning
- Error Threshold - Select the value from the drop-down on when you want to be notified when there is an error. The first drop down is to select the Operator (> or =) and the second operator is to select the warning value. For e.g., When the number of service instances are > 20, throw an error
- Send Instance Detail In Notification - Selecting this checkbox will send the instance details and exception description in the notification email when the number of instances exceeds the warning and/or error threshold level
- Send message content and context in the notification email - Selecting this checkbox will send the detailed message content and message context in the notification email when the number of instances exceeds the warning and/or error threshold level
- How often you want to monitor - Select the duration of how often you want to monitor for suspended service instances. The options are - Daily, Weekly, and Monthly. Depending on the selection, you can add further filter criteria on how often you want to monitor
- Daily - will set up the data monitor to execute everyday
- Weekly - The data monitor will execute on the selected days of the week (Every Day, Monday-Friday, or the selected day of choice). To add a single day of the week, click the control and select the day of your choice
- Monthly - The data monitor will execute on the selected day of the month. To add a specific day of the month, enter the date in the space provided or select the date from the drop-down
If a particular day is not present in the month (say, 29th Feb), then the system will skip that month and generate the results on the specified day in the next month
- When during the day - Select the time when the data monitor must execute on a particular day during the business hours. You can customize your business hours by selecting the values from the Business Day Start and Business Day End options
By default, the business day start time is 9 AM and business day end time is 5 PM
Depending on the business hour selection, the following options are available when you want the data monitor to execute on a particular day.
- At the end of the business day - If this option is chosen, the data monitor will execute depending on the value chosen in the Business Day End field. For e.g, if the business end time specified is 5 PM (default value), the data monitor will execute exactly at 5 PM
Monitor every X minutes/hours - You can define specific intervals during the business hours when the data monitor should execute. The options available are Minutes and Hours
- Minutes - Select the number of minutes after which the data monitor should execute. E.g., 30 minutes means the data monitor will execute once every 30 minutes.
- The minimum time interval supported is 15 minutes, which means the data monitor will execute 4 times (15 minutes x 4 = 60 minutes) in an hour.
- Hours - Select the number of hours after which the data monitor should execute during the business hours. E.g., 2 hours means the data monitor will execute once in 2 hours between 9 AM and 5 PM.
- The minimum time interval supported is 1 hour.
- Advanced - This section can be used to define advanced settings for data monitoring. You can define a date range (from date and to date) when you want the data monitor to execute. You can set up a data monitor to start, say, on a Monday and end on the next Tuesday. In addition to this, you can set up the other settings mentioned above in the previous steps. This means the data monitor will only start to run next Monday and will not execute after Tuesday
By default, the Advanced section will be in Collapsed state. Click the '>' button to expand the Advanced section and make the settings.
- Based on only Query result count (no date/time filter) - Selecting this option will execute the query without applying any date time filter. i.e All the data will consider for SQL query execution.
- Based on Query result with date/time range - Selecting this option will apply the Creation Time filter for SQL query execution. ( The values configured for setting trigger time, Set Monitoring Times ->At What Time, will be used for Creation Time filter ). For eg: When Monitoring set frequency is set for 1 hour, then for every SQL query execution the query will consider only 1-hour data "[DateCreated] > DATEADD(HOUR, -1, GETDATE())".
- Is Action Required - Selecting this checkbox enables you to automate tasks such as regularly terminate instances, resume instances and so on. You can define an action such as "Whenever there are Suspended (Non-Resumable) service instances during the monitoring time range, the system should automatically terminate them. Similarly, whenever there are Suspended (Resumable) instances, the system can automatically Resume these messages. The following options are available for the Is Action Required setting:
- When to Action - Defines when you need to take an action - Always (or) when there is an error condition (or) a warning condition. You need to be cautious about enabling it since some of the actions performed by the system cannot be undone
- What Action - Defines what action needs to be taken - Resume, Suspend or Terminate. You need to be cautious about enabling it since some of the actions performed by the system cannot be undone
When you select the action as "Terminate", you will notice the "Archive message(s) content & context associated before Terminate" checkbox. Selecting the checkbox will automatically back up the service instance details to a folder and then terminate the instance. See the below "Archive and Terminate Suspended Service Instances" section to configure the archive location
- Number of Retries - Click the numeric value to set the number of retries the process monitoring service should attempt
- Summary - The summary information will automatically be populated based on the information selected in the previous columns
- Save & Close - Once all the values are selected in the previous steps, click Save & Close to save the settings
- Save & Add Another - The entered values will be saved and a new screen will be displayed to create another data monitor
- Close - Click Close to discard the changes. When the close button is clicked, the system will display a confirmation message as "There are unsaved changes, do you still want to close?". Click Yes to close the Data Monitoring creation window. Click No to remain on the same page
- General guideline - Provides information about the operations that can be performed in the current screen
Archive And Terminate Suspended Service Instances
There may be occasions when a BizTalk administrator wants to terminate service instances. Once a service instance is terminated, there is no way the message information can be retrieved from BizTalk. Therefore, a message once deleted is forever deleted from the system.
However, there may be situations when the administrators would like to access a previously terminated message. To overcome this limitation, BizTalk360 provides the option for administrators to be able to archive the service instance information (Message Content & Context) before terminating the service instances. In this section, we are referring to the Data Monitoring feature with the capability to archive the message before terminating them.
Configuring The Archive Location For Data Monitoring
The first step in setting up the "Archive before Terminate" option in Message Box Data Monitoring, is to configure the location where the archived files are to be stored. You need to specify a valid file location in order to be able to archive the service instance information before terminating them.
- Navigate to BizTalk360 Settings
- Click System Settings option from the left menu
- In the Archive location for data monitoring (archive before terminate option), enter the path where you want to archive the service instance information
- Click Save to save the information
Once you have configured the Archive location, you can set up a Message Box Data Monitor and have it archive service instance information.
- In the MessageBox Data Monitor, enter the details in the fields as appropriate. Refer to this section to understand about each field in the message box data monitor creation page
- In the Set Actions section, select the Is Action Required checkbox
- Select When to action option from the drop-down (depending on the requirement) - Every Time, On Error Condition, On Warning Condition
- In What Action option, select 'Terminate' as the option from the drop-down. When you select Terminate, a checkbox will appear as "Archive message(s) content & context associated before Terminate". Select the checkbox to activate the archive before terminate option
- Click Save & Close to save and close the data monitor
Data Monitoring Dashboard - Action taken
When the data monitor executes at the specified time (as defined in the data monitor), and if there are any suspended service instances, they will be archived into the specified path (as mentioned in the System Settings screen) and then terminated. This action will be reflected in the monitoring dashboard. If the operation happened successfully, you will notice the 'Download Archive' column header with the Download link to the service instance information. You can download the specific service instance-related information from the data monitoring dashboard. Whenever an action is taken on the service instance, a Gear Icon will be visible in the monitoring dashboard.
The file path will contain the archived service instances (the .out file and .xml file). You will also find the zipped version of these files just in case you need to transfer these files to your team members.
MessageBox Data Monitoring Email Notifications
Once the MessageBox Data Monitor is created and configured, it will execute during the specified time intervals. The Data Monitoring dashboard will display the status of the MessageBox data monitor on a specific date at a specific time interval in the calendar chart control.
Based upon how you configured your Data Monitoring alarm, emails might get triggered to the email id(s) which are specified while creating that Data Monitoring alarm.
The email alert(s) will contain the Message Box Data Monitor Execution Results of the data monitor. By default, if you only selected the 'Use this alarm for Process Monitor Alerts' option, you will only receive the Failure emails to the configured email address(es). Alternatively, if you also selected the 'Notify on Success as well' option, you will receive the success alert emails as well to your mailbox. The following screenshot depicts a success email notification received for a configured MessageBox data monitor.
The email notification provides the detailed status of the data monitor - the status of execution, the warning & error count as defined in the data monitor and the actual count of service instances that were in the particular state when the data monitor executed. The email notification also provides the detail of whether any specific action was taken by the data monitor (suspend/terminate/resume) as defined in the data monitor.
In-Depth Look At Message Box Data Monitoring Dashboard
The Data Monitoring dashboard will display the status of service instances in a graphical manner. The data monitor status information will be plotted against the corresponding time on the calendar chart control. The status information is color-coded based on the actual status of the data monitor. Let's assume the scenario - the administrator wants to terminate the Suspended (Non-Resumable) messages when there are more than 20 instances at a given time, and receive a warning when the count of service instances is greater than 15. In this case, when the number of suspended service instances is, say, 14 during a particular time period, the status will be plotted in Green color on the calendar chart control. Similarly, when the number of suspended service instances is, say, 17 during a particular time period, the status will be plotted in Yellow color on the calendar chart control indicating a warning & the notification will be triggered to the administrator's email. When the number of suspended service instances is, say, 25 during a particular time period, the status will be plotted in Red color on the calendar chart control indicating a breach in the error threshold condition. When the status message is clicked, the system will display a pop up with the actual status information of the data monitor.
The Data Monitoring dashboard works based on the time set in the System Settings. Click the System Settings link in the application to set the date and time format and select the appropriate time zone based on your preference. Once set, the setting will be applicable throughout the application.
- Calendar Chart Area - The status information of the data monitor will be plotted in this section. The information will be plotted against the time interval on the Y-axis. Say, for instance, a data monitor is set up to monitor for suspended non-resumable service instances once every 15 minutes during the business hours from 9 AM to 5 PM. In this case, the points will be plotted at every 15-minute slot starting from 9:00 AM through 5:00 PM. Depending on the status of the data monitor, the points will be plotted with either Green/Red/Yellow color code tick mark against the data point.
In the above screenshot, you will notice data points plotted with a green tick mark, yellow tick mark, and a red tick mark. A green tick mark indicates that the warning/error threshold condition was not met and everything was normal during the time period. A red tick mark indicates the vice-versa condition - the error threshold condition was met (when the actual count of suspended non-resumable service instances is greater than the configured error threshold value). A yellow tick mark indicates that the warning threshold condition was met (when the actual count of suspended non-resumable service instances is greater than the configured warning threshold value and less than the error threshold value).
You will notice a small 'Gear' icon on the data point that is plotted in the calendar chart area. This icon signifies the "Is action required" functionality of message box data monitoring. Depending on the "is action required" settings, if any specific action is taken, you will notice the gear icon on the data point that is plotted in the calendar chart area.
- Calendar Control - The calendar control can be used to pick a custom date and view the data monitor information. When a date is clicked, all the data monitors configured for the particular date are listed in the calendar chart area. For easy identification, the dates that have data monitor configured are color-coded as Red, Green, and Grey
Red - At least one of the configured data monitors on the specific date has failed to meet the configured criteria
Green - All the configured data monitors on the specific date has successfully met the configured criteria
Grey - Indicates the current date
- Expand/Collapse Calendar - Click the Collapse Calendar link to collapse the calendar control and get a full view of the calendar chart area. Once the calendar is collapsed, click the Expand Calendar link to display the calendar control
- Message Box Data Execution Results - Click the plotted data point on the calendar chart area to view the Message Box Data Execution Results
The pop-up window will display the following information:
Start Time - The starting time of the window when the data monitor executed
End Time - The end time of the window when the data monitor executed
Overall Status - Displays either Success Failure or Warning depending on the status of data monitoring execution
Task Action Taken - Displays the status of whether any task action has been taken or not by the data monitor. If any action has been taken, a green tick mark appears. If no task action has been taken, a red cross mark will appear in this area
Warning Threshold - The count displays the warning threshold value set in the "Set the notification condition" section while setting up the data monitor
Error Threshold - The count displays the error threshold value set in the "Set the notification condition" section while setting up the data monitor
Actual Count - The count displays the actual count of service instances in a particular state (resumable/non-resumable) during the specific time period
Configured Query - Displays the information about the query that is configured to execute along with the data monitor. This section displays the filter, the corresponding operator, and value configured for the query
Task Action Result (tab) - This section will be visible only if the "Is action required" section is configured in the data monitor. Depending on the action that is taken by the data monitor, the results will be populated within the following column headers -
- Status - Displays the status of the action that has been performed. Displays the status result as Success or Failure
- Instance Name - Displays the name of the service instance
- Action When - Displays when the action has been taken
- Action Command - The action that has been performed on the service instance
- Exception - If there are any exceptions while taking the action, the exception information will be populated here
- Filter - This option helps to filter out the data monitor results of other sections (such as ESB/EDI/BAM/Tracking Data/Process monitoring) from the dashboard. This helps to give a clear view of the data monitoring results
- Refresh - Click the Refresh link to refresh the calendar chart area with the latest updates
Service Instances Action Based On Retry Count
You may face a situation there may not be any action on suspended instances when you configured automate action in Message Box Data Monitoring. BizTalk360 Data Monitoring schedule results show that the task action is successful but no action has been taken.
BizTalk360 will take action the instance suspended on the artifact irrespective of the "Number of Retries" count set in the schedule. If the Instance is suspended in port then BizTalk360 will check how many times the instance is retried by BizTalk Server.
If the number of retries done by BizTalk is less than or equal to the count set in BizTalk360 then it will resume the instance until it exceeds the count set. If the number of retries done by BizTalk is greater than to the count set in BizTalk360 then it will stop resuming the instance on further monitoring schedule.
Check the retry count between the BizTalk artifact and in the BizTalk360 count in the UI. If the BizTalk360 Configured Value is 3 and the BizTalk server's Current Retry Count is 4 so the BizTalk360 didn't take any action.
Please increase the number of retries count and check if it is resuming the instances from next time.