PMN Smoke Test - Pipeline Test Traceability
Step | Description | Automated By | Notes | |
|---|---|---|---|---|
1 | Log in | Log into the updated portal | Nearly all tests |
|
1a |
| Check that previous requests are displayed (in Production only). | Not tested |
|
|
| Review that past metadata, request parameters, DataMart routings, and results are present. | Not tested |
|
|
| Staging may not be using the most recent backup of the database and therefore the queries may not match up with what is seen in production | Not tested |
|
2 | Download and install the oldest supported DataMart Client | Download and install the oldest supported version of the DMC from Offline DataMart Client Installer (required DMC updates will be noted on that page with the targeted network). | Not tested |
|
|
| If this release is a required DMC upgrade please test the most current version of the DMC. | Not tested |
|
4 | Check DataMart network connections | The DataMart Client should connect to all networks running the same version of the software. Some updates to the DataMart Client may also connect to networks using a prior version. | Not tested |
|
|
| If you previously were connected to networks, check that the settings transferred and that you are currently connected and that requests are displayed. | Not tested |
|
|
| If you were not previously connected to a network, connect now. Check that you can connect and that requests are displayed. | Not tested |
|
5 | Check that all DMC adapter versions automatically download | Go to the DMC "adapter processor packages" page and delete all currently downloaded adapters. Attempt to open a request that was created using the current as well as previous versions of PMN, in order to ensure that all versions of the adapter download and function. | GetAdapterPackage_ReturnsPackage, | Note that for "GetAdapterPackageForRequest" tests require the IDs of several requests, along with the versions they targeted. |
6 | Submit requests | 1 request per model processor used on the network should be submitted to a test DataMart. Remove rows below as necessary. | RespondToRequest_ReUploadResponse_HttpClient_ShowsResultsModified, | Requests submitted as part of each of the three referenced tests |
6a |
| Submit a QE File Distribution request (use a zip file) (PCORNET only) | Not tested |
|
6b |
| Submit a QE Modular Program distribution request (use a zip file) (Sentinel and HDC only) | (See step 6) |
|
6c |
| Submit a Menu-Driven Query request (PCORNET only) | Not tested |
|
7 | Placing requests on Hold | View the request you had submitted in part 6 in the DataMart Client. | PutRequestOnHold_UsingMockDmc | Uses a test mock to simulate traffic from the DMC |
|
| Select the 'Hold' button | (See step 7) |
|
7a |
| Status successfully changes to 'On Hold' | (See step 7) |
|
7b |
| Return to the request and select the button to remove the Hold | RemoveHoldFromRequest_UsingMockDmc | Uses a test mock to simulate traffic from the DMC |
|
| You are then able to upload results. | Not tested | No technical reason this could not be implemented, it just was not at the time. Should not require more than a day or two to implement. |
|
| Status successfully changes to completed | Not tested |
|
8 | Re-Uploading results | Select a request with a status of completed (one that you've already uploaded results for) | RespondToRequest_ReUploadResponse_HttpClient_ShowsResultsModified | Uses a test mock to simulate traffic from the DMC |
|
| Add an additional file to upload | (See step 8) |
|
8a |
| Select the new file and select re-upload results | (See step 8) |
|
|
| Attempt to re-upload with and without adding a comment | (See step 8) |
|
|
| Re-upload the result | (See step 8) |
|
|
| Review the status of the request and ensure it says 'Results Modified' | (See step 8) |
|
9 | Check email notifications | Check that you received the New Request Submitted notifications for the requests that you submitted | Not tested |
|
10 | Respond to the requests | Respond to each of the requests that you submitted above. Remove rows below as necessary. | ModularProgramRequest_RespondWithOneZipFileUsingMockApi_VerifyResults |
|
10a |
| Download the QE File Distribution request file from the DMC, unzip and open the test file, and respond to the request (PCORNET only) | Not tested | PCORNET out-of-scope |
10b |
| Download the the QE Modular Program request file from the DMC, unzip and open the test file, and respond to the request. (Sentinel and HDC only) | ModularProgramRequest_RespondWithOneZipFileUsingMockApi_VerifyResults | Includes step 11b |
10c |
| Respond to the Menu-Driven Query request (PCORNET only) | Not tested | PCORNET out-of-scope |
|
| export MDQ results from DMC and check excel for at least one routing | Not tested | PCORNET out-of-scope |
11 | View results on the portal | Check the results for each request. Remove rows below as necessary. | ModularProgramRequest_RespondWithOneZipFileUsingMockApi_VerifyResults | Tested as part of step 10b |
11a |
| Check results for the QE File Distribution request (PCORNET only) | Not tested | PCORNET out-of-scope |
11b |
| Check results for the QE Modular Program Distribution request (Sentinel and HDC only) | ModularProgramRequest_RespondWithOneZipFileUsingMockApi_VerifyResults | Tested as part of step 10b |
11c |
| Check results for the Menu-Driven Query request (PCORNET only) | Not tested | PCORNET out-of-scope |
|
| Download and check aggregate results csv and excel |
|
|
|
| Download and check individual results csv and excel |
|
|
|
| PCORNET only |
|
|
12 | Submit a registration | Log out of the portal or navigate to the portal in another browser. Submit a new registration. | SubmitAndApproveUserRegistration | Automation includes 2 additional tests: |
13 | Approve the registration | Log back in as the system administrator and approve the new user. Ensure the user is subscribed to the new request submitted notification | (See step 12) | Automation includes 2 additional tests: |
14 | Log in as the new users | Login as the new user. Check that the query tool home page loads | (See step 12) |
|
14a |
| Check that you see new request submitted notifications in the notification panel | Not tested |
|
15 | Check the Forgot Password link | Log out as the new user and on the login page check to make sure you are able to click and submit to the Forgot Password hyperlink. |
|
|
16 | Install and check the network connections have carried over correctly for the new version of the DMC | IF this is not a required DMC update release: | Not tested |
|
|
| Install the most recent version of the DMC and make sure the network connections have carried over appropriately when starting the DMC up after install. | Not tested |
|
17 | Check DMC Authentication Tracking | (only for DMC version 2020.2 or newer) Navigate to web portal profile page of user who opened most recent DMC. Verify that DMC authentications are tracked, and include correct DMC version number. | Not tested |
|