Wednesday 20 April 2011

Custom Extraction Rules for Visual Studio Load Tests - Dealing with Dynamic Control Identifiers

I was recently commissioned to create and fix up several Visual Studio 2008 Load Tests for one of my clients. There were some questions raised about the validity and accuracy of the load test results - so I was brought in to do a "Load Test Audit".

I started by creating some data validation scripts which confirmed that data was being updated correctly. Unexpectedly, even though the tests were running without failure, the number of records affected in the database was completely different to the anticipated results.

Upon investigation, I found several issues which caused the Load tests to intermittently or "silently" fail (i.e. tests passed, but exceptions were logged in the backend of the custom application). There were several problems I found. I cover two of them below:

One of the problems is that one of the buttons in the recorded webtest would sometimes not be correctly triggered. Consequently all subsequent web test requests in that Load Test scenario would fail.

Turns out, there was a link button in one of the ASP.NET Web Parts rendered as part of a GridView - and the id of the control would actually change based on the number of records included in the grid.

Consequently, in most cases there was 1 record in the grid so the "lnkAdd" button was "lnkAdd1". This matched what was recorded in the Visual Studio Webtest. However, the load test would fail when there were a larger number of records as Visual Studio couldn't find the "lnkAddX" button.

To resolve this, the simplest way to dynamically determine the id of the dynamic control was to use a custom extraction rule that sets a context value for use in subsequent requests. Like so:
public class GetAddButtonId : ExtractionRule
        /// Add button name. TODO: Add defensive code
        /// /// public override void Extract(object sender, ExtractionEventArgs e)
            e.Success = true;
            if (!string.IsNullOrEmpty(e.Response.BodyString))
                Match value = Regex.Match(e.Response.BodyString, @"ctl00\$ContentPlaceHolder1.[^>]*?grdScheduleChangeResults\$.[^>]*?\$lnkAdd");
                if (value.Success)
                    this.ContextParameterName = "AddButtonId";
                    if (!e.WebTest.Context.ContainsKey("AddButtonId"))
                        e.WebTest.Context.Add("AddButtonId", value.Groups[0].Value);
                        e.WebTest.Context["AddButtonId"] = value.Groups[0].Value;

I added this Custom Extraction Rule as part of the web test, updated subsequent requests to use the Context value for the control name - and the issues were resolved.

There were also issues with the Validation rules in the Load Test - as the tests were running without failure, but they were actually just hitting the CustomError.aspx page (this occurs when there's an exception). I had to add a new Web Test Validation rules as below:

public override IEnumerator GetRequestEnumerator()
            if ((this.Context.ValidationLevel >= Microsoft.VisualStudio.TestTools.WebTesting.ValidationLevel.High))
                if (!Context.ContainsKey("IgnoreErrors") || (Context["IgnoreErrors"].ToString() != null && bool.Parse(Context["IgnoreErrors"].ToString())))
                    ValidationRuleFindText validationRule2 = new ValidationRuleFindText();
                    validationRule2.FindText = "Unable to perform operation";
                    validationRule2.IgnoreCase = true;
                    validationRule2.UseRegularExpression = false;
                    validationRule2.PassIfTextFound = false;
                    this.ValidateResponse += new EventHandler(validationRule2.Validate);

                    ValidationRuleFindText ErrorSummary_Validation = new ValidationRuleFindText();
                    ErrorSummary_Validation.FindText = "ErrorSummary";
                    ErrorSummary_Validation.IgnoreCase = true;
                    ErrorSummary_Validation.UseRegularExpression = false;
                    ErrorSummary_Validation.PassIfTextFound = false;
                    this.ValidateResponse += new EventHandler(ErrorSummary_Validation.Validate);

                    //'An open change request already exists

                    //Wasn't detecting custom error page as issue in test.
                    ValidationRuleFindText customError_Validation = new ValidationRuleFindText();
                    customError_Validation.FindText = "CustomError.aspx";
                    customError_Validation.IgnoreCase = true;
                    customError_Validation.UseRegularExpression = false;
                    customError_Validation.PassIfTextFound = false;
                    this.ValidateResponse += new EventHandler(customError_Validation.Validate);

                    ValidationRuleFindText callback = new ValidationRuleFindText();
                    callback.FindText = "Invalid postback or callback argument";
                    callback.IgnoreCase = true;
                    callback.UseRegularExpression = false;
                    callback.PassIfTextFound = false;
                    this.ValidateResponse += new EventHandler(callback.Validate);

                    this.StopOnError = true;
            return null;
Hope this helps someone in the future when they are troubleshooting their Visual Studio Web Tests or Load Tests!


Friday 8 April 2011

DDK's Guide to the Estimation of IT Projects

As an Architect and the .NET Principal at Oakton NSW, I have to do my fair share of Estimates and Proposals. I am also often asked to review and revise other peoples estimates - to "Quality Stamp" them so to speak. There are some common things that I pick up on - hence the driver behind this blog post.

Summary Diagram of the DDK Estimation Technique:

Here are some of the important things you should consider when developing estimates:

1. Estimate from the bottom up rather than from the top down. The focus and detail of this approach helps you to substantiate your estimates to others and show you've used due diligence in arriving at your estimate. A detailed function point analysis (FPA) is ideal when trying to minimise risk as much as possible (especially for fixed cost projects).
2. There are 2 critical variables which can make a project take much longer than expected and estimated. If you have these components, you need to increase your estimate to more than you expect:
  a. The larger and more complex the project, the more likely it is to take longer than expected.
  b. The more new technologies or new techniques involved, the more likely it is to take longer than expected

3. Communicate with and update the client regularly – Don’t be afraid to re-estimate your tasks and let the client know if things will take longer or shorter than expected. The earlier they know, the earlier corrective action can be taken.

4. Larger projects are harder to estimate. Only estimate small components of a project if possible – don’t estimate all releases. Deliver and estimate in increments if the client/contracts allow.

5. Make sure you consider the following components in your Estimation Checklist before giving it to the client:
  a. End User Documentation
  b. System Documentation
    i. Rollback Plans
    ii. Non Functional Requirements (NFR)
    iii. Technical Design & Specifications
    iv. Functional Design & Specifications
    v. Establishing Metrics (e.g. what performance is expected on what servers and with what data load)
    vi. Test Plans
    vii. Test Scripts
  c. Testing
  d. Training and Change Management (you can't just give someone an application and expect them to start using it effectively!)
  e. User Acceptance Testing
  f. Integration Testing (esp when integrating with Legacy Systems)
  g. Deployment Activities and Productionizing Systems
    i. Especially with complex deployments. These deployment activities are typically ongoing rather than once off.
  h. Meetings Drag Factors such as regular Meetings and Discussions. Team leaders and architects need a drag factor SCRUM meetings and requirement gathering meetings.
    i. Triage Meetings
    ii. Code Reviews
    iii. Level 2 Reviews by Testers e.g. For one of our projects, it took an average of 45 minutes per TFS work item/ticket.
  i. Focus Groups
  j. Licenses
  k. Configuration
    i. Third Party Components
    ii. Firewall Configuration
    iii. Database Configuration
  l. Data Migration
  m. Handovers (Including Warranty Periods)
  n. You need to anticipate who is developing it.
    i. Offshore Models/Engagements – have to spend 2-3 times the effort in non-development activities such as coordination, “hand-holding” and quality assurance when your team is overseas. You also need to spend much more time on specification documents to avoid communication issues.
  o. Temper your Optimism by considering different scenarios
    i. Best Case
    ii. Worst Case
    iii. Likely Case Scenario
  p. If uncertainty on a project is High, add an uncertainty multiplier to your estimate
  q. Don’t estimate more than 8 hours per day.
  r. Budget time for Performance Testing
    i. Time for Load Tests

6. If possible, do a Proof of Concept (PoC) before providing the estimates (or just estimate the PoC) – especially when using new combinations of technologies.

7. Learn from Historical Data. Try to learn from similar projects and how the estimates compared with the actuals. Use your company portal to discover estimates and ask other people in your company for similar estimates that they did.

8. Sanity Check your estimates. i.e. Have your estimates peer-reviewed to help you ensure consistency and coverage in your estimates.

9. Cross-Check your estimates. If you can convince all stakeholders that the estimate is valid and establish buy-in to that estimate, you have the basis of a good estimate.

10. Don’t underestimate Non-Programming/Infrastructure activities.

11. Don’t change estimates if possible if they are based on a solid agreement or understanding. Instead, try and change the commercial arrangements surrounding the estimate. E.g. don’t change the estimates unless they are proven unreasonable – change the rate if possible.

12. All estimates are guesses – try and reduce the uncertainty – but you cannot remove uncertainty completely.

13. Try to make your estimates from a fully informed standpoint - the same as a General shouldn't make strategic decisions in the fog of war. Request and Review as many materials (including scribbled diagrams and requirements documents) as time allows. The more you discuss your understanding of the project and talk to the end users, the more likely you are to make an informed estimate. Again, this allows you to back up your estimates to all stakeholders. If there are factors in the project that are particularly unclear or uncertain, add an "uncertainty" multiplier on the item. Encourage the client to help you clear up this uncertainty if possible.

14. Use the right tools. Microsoft Project is a good start. Learn how to use it properly - plus it can then create and update developer work items for you in Microsoft Team Foundation Server (TFS)  when you are ready to start work.

15. Don't forget that external dependencies (e.g. a 3rd party is creating web services for you, waiting on documentation) will slow the project down. Ideally start the project when work you depend on is complete - otherwise you'll need to factor downtime into your budget, estimates (an expected "downtime" item) and estimate assumptions.

Any thoughts or comments on this guide and checklist are welcome - I will update the list based on feedback.


Thursday 7 April 2011

I'm now a SharePoint 2010 Microsoft Certified Professional Developer (MCPD)

I recently passed my 2nd SharePoint 2010 exam (70-573 and now 70-576) - so I'm now officially a SharePoint 2010 MCPD!

Next step is MCITP (covering the Infrastructure side of the SharePoint equation) and then onto the coveted Microsoft Certified Master (MCM) certification.

The MCM is apparently a challenge to get (involving panel interviews by other MCMs and 3 weeks of training) - and there is only one so far in Australia.