Tuesday, May 31, 2011
Monday, May 30, 2011
BizTalk Talk (Speech) Adapter - Screencast and download
Thursday, May 26, 2011
Wednesday, May 25, 2011
Checklist: Monitoring BizTalk Server with Operations Manager 2007
Checklist: Monitoring SQL Servers
Checklist: Increasing Availability with Disaster Recovery
Backing Up and Restoring BizTalk Server Databases
How to Configure the Backup BizTalk Server Job
How to Configure the Destination System for Log Shipping
How to Restore Your Databases
Backing Up and Restoring BAM [For future reference only]
Monday, May 16, 2011
Processing Inter-dependent files using a Non-Uniform Sequential Convoy - Kent
BizTalk supports messaging patterns that provide the ability to wait for messages to arrive before completing a business process(or orchestration). These mechanisms tend to use correlation and the use of convoys. Stephen W Thomas has written a whitepaper that dives into these topics further.
The second artifact represents data that could be generated by an upstream system such as SAP or a Mainframe. This document is entirely fictitious so don't look too much into it. Also note that I have a promoted property called FileNamethat is used in my Correlation Type.
Below is a snapshot of what our orchestration looks like.
A few things to note is that Non-Uniform Sequential convoys to require that both receive shapes connect to the same receive port. This logical receive port also needs to be marked for Ordered Delivery.
For the initial receive shape we need to set a few properties. Since this is the first Receive shape we need to set theActivate property to True in order to instantiate the orchestration.
The other property that we need to populate is the Initializing Correlation Sets. In order for BizTalk to "wait" for the work file to be picked up by the same instance of the orchestration that consumed the signal file we need to initialize a correlation set. (Keep reading for more info on how to create the Correlation Set).
Prior to creating a Correlation Set, you need to create a Correlation Type. You can do this from the Orchestration Viewtab.
We then need to create a Correlation Set, which is instantiated by the initial Receive Shape, that is based upon the Correlation Type that was just created. Correlation Sets are also configured in the Orchestration View tab.
In the Rename Work File Expression shape we are going to call a .Net Helper class that will aid in renaming work file in the source folder. This step is a critical step in the process. Since the work file is completely written before the sig file is, we cannot use the original file extension in the Receive Location file mask. Otherwise, this would prompt BizTalk to consume the work file prior to us wanting it to be consumed. By appending a temporary extension, like .BIZ, to the end of the work file name, we can be assured that BizTalk will pick the file up when we want it to. So in the Receive Location we will use a *.BIZ extension instead of a *.XML.
Note that I have hard coded the path of the source location. Since this is a POC, this is ok but this would not be a suitable solution for a production environment
In order for BizTalk to "wait" for the work file to be picked up, we need to set the Following Correlation Sets property with the same Correlation Set that we initialized in the first Receive shape. Since the rename operation occurs the step before, the "wait" time will be extremely small. The main point here is that we want to control when BizTalk picks up the work file. It is essentially the rename operation and the second receive shape/location that controls this.
In order to simulate how an upstream system would write the files, I drop a file into the work folder with the original extension. I then drop a sig file into the sig folder. The sig file will get picked up and BizTalk, via the .Net Helper, will rename the work file. The receive location, for work files, will then pick up the work file and the orchestration will finish processing both files.
Through the use of a Messaging Pattern and with the help of a .Net Helper class we can process a set of files in a known order.
Wednesday, May 11, 2011
Archiving and Purging the BizTalk Tracking Database
While previous versions of BizTalk Server included sample scripts for archiving tracked messages and purging the BizTalk Tracking database, BizTalk Server automates both processes using the DTA Purge and Archive job. By archiving and purging data from the BizTalk Tracking database, you can maintain a healthy system, as well as keep your tracking data archived for future use. Because BizTalk Tracking database archives accumulate over time and consume disk space, it is a good idea to move the BizTalk Tracking database archives to secondary storage on a regular basis.
When you purge data from the BizTalk Tracking database, the DTA Purge and Archive job purges different types of tracking information such as message and service instance information, orchestration event information, and rules engine tracking data.
The age of a tracking data record is based on the time the tracking data was inserted into the BizTalk Tracking database. The DTA Purge and Archive job uses the time stamp to continuously verify whether the record is older than the live window of data. After every live window period, the BizTalk Tracking database is archived and a new archive file is created. At each SQL Server Agent job interval specified by the job schedule, all completed tracking data older than the live window period is purged.
BizTalk Server uses the concept of a soft purge and a hard purge. The soft purge is used to purge completed instances, while the hard purge is only used to purge incomplete instances.
In the DTA Archive and Purge job, the sum of the LiveHours and LiveDays parameters is the live window of data you want to maintain in your BizTalk Server environment. All data associated with a completed instance older than this live window of data is purged. By default, the DTA Archive and Purge job is not enabled. You must first configure and then enable the job.
For example, you can configure the DTA Purge and Archive job to run every 20 minutes, and set LiveHours=1 and LiveDays=0. The first time this SQL Server Agent job runs (T0), it takes a backup of the tracking database by creating an archive and an entry is saved in the database with this timestamp. A successful archive is necessary in order to purge tracking data. If the archive was successful, then all the data associated with the instances that completed over an hour ago is purged. Each time the job runs, completed data over one hour old is purged. On the 3rd run (after one hour), a new archive is created that contains the data for all instances that were inserted into the tracking database in the last one hour segment.
Here is how you would configure the Archive and Purge step in the DTA Purge and Archive job to match the example above:
Soft purge process
Because the soft purge only purges data associated with completed instances, if you have many looping instances that run indefinitely, then your tracking database would grow and these instances would never be purged. The hard purge date allows all information older than the specified interval to be purged except for information indicating a service's existence. You set the hard purge using the @nHardDeleteDays parameter in the Archive and Purge step in the DTA Archive and Purge job. The hard purge setting should always be greater than your soft purge setting. In other words, @nHardDeleteDays should be greater than the sum of @nLiveHours and @nLiveDays.
Archiving and purging includes the features described in the following table:
|Hard purge||Enables you to configure a time interval to purge information for incomplete instances older than a specified date.|
|Copying tracked messages to tracking database||Using the CopyTrackedMessageToDTA option, you can directly copy tracked messages from the MessageBox servers to your BizTalk Tracking database. This is required in order to purge data using the DTA Purge and Archive job.|
|Archive validation||Enables you to optionally set up a secondary database server to validate the archives as they are created.|
|Tracking support for multiple BizTalk Tracking database versions||Enables you to use tracking support with BizTalk Server 2004 and BizTalk Server 2006 database archives.|
|Reduction of tracking data||Substantially reduces the amount of tracking data stored without reducing any tracking information generated. This results in slower growth of the tracking database.|
|Faster tracking operations, significant optimization in database schemas||Enables you to use tracking tasks for finding messages and service instances on large databases; this feature has been significantly optimized.|
In This Section
- Checklist: Archiving and Purging the BizTalk Tracking Database
- How to Configure the BTS_BACKUP_USERS Role for Archiving and Purging Data from the BizTalk Tracking Database
- How to Configure the DTA Purge and Archive Job
- How to Purge Data from the BizTalk Tracking Database
- How to Manually Purge Data from the BizTalk Tracking Database
- How to Enable Automatic Archive Validation
- How to Copy Tracked Messages into the BizTalk Tracking Database
- Improving the Performance of the Archiving and Purging Process
- How to Turn Off Global Tracking
Monday, May 9, 2011
SharePoint Adapter in BizTalk Server
19. In SharePoint Site URL type http://localhost
drop XML file to input folder and check your SharePoint list for appeared records.
BizTalk SSO Configuration Data Storage Tool
If you’ve been in the BizTalk world long enough, you’ve probably heard that you can securely store name/value pairs in the Enterprise Single Sign-On (SSO) database. However, I’ve never been thrilled with the mechanism for inserting and managing these settings, so, I’ve built a tool to fill the void.
Jon Flanders did some great work with SSO for storing configuration data, and the Microsoft MSDN site also has a sample application for using SSO as a Configuration Store, but, neither gave me exactly what I wanted. I want to lower the barrier of entry for SSO since it’s such a useful way to securely store configuration data.
So, I built the SSO Config Store Application Manager.
I can go ahead and enter in an application name, description, account groups with access permissions, and finally, a collection of fields that I want to store. “Masking” has to do with confidential values and making sure they are only returned “in the clear” at runtime (using the SSO_FLAG_RUNTIME flag). Everything in the SSO database is fully encrypted, but this flag has to do with only returning clear values for runtime queries.
You may not want to abandon the “ssomanage” command line completely. So, I let you export out the “new application” configuration into the SSO-ready format. You could also change this file for each environment (different user accounts, for instance), and then from the tool, load a particular XML configuration file during installation. So, I could create XML instances for development/test/production environments, open this tool in each environment, and load the appropriate file. Then, all you have to do is click “Create.”
If you flip to the “Manage” tab of the application, you can set the field values, or delete the application. Querying an application returns all the necessary info, and, the list of property names you previously defined.
If you’re REALLY observant, and use the “ssomanage” tool to check out the created application, you’ll notice that the first field is always named “dummy.” This is because if every case I’ve tested, the SSO query API doesn’t return the first property value from the database. Drove me crazy. So, I put a “dummy” in there, so that you’re always guaranteed to get back what you put in (e.g. put in four fields, including dummy, and always get back the three you actually entered). So, you can go ahead and safely enter values for each property in the list.
So how do we actually test that this works? I’ve included a class, SSOConfigHelper.cs (slightly modified from the MSDN SSO sample) in the below zip file, that you would included in your application or class library. This class has the “read” operation you need to grab the value from any SSO application. The command is as simple as:
Finally, when you’re done messing around in development, you can delete the application.
I have plenty of situations coming up where the development team will need to secure store passwords and connection strings and I didn’t like the idea of trying to encrypt the BizTalk configuration file, or worse, just being lazy and embedding the credentials in the code itself. Now, with this tool, there’s really no excuse not to quickly build an SSO Config Store application and jam your values in there.