SAP Items #7 & #8: Pass 2 for Breaks in Enrollment & Summer Drops
UAT Overview
Per the 9/14/2022 Financial Aid Maintenance List from FAC on Satisfactory Academic Progress change requests, a solution was requested to:
- Enable the FA SAP Process to read a prior term FA SAP Student Status when there is a break in enrollment and generate an amended SAP Status based on a combination of current Calculated SAP Status and Prior Term SAP Status.
- Handle "Summer Drops" so the system does not read the SAP Status from the Student SAP page for a term where the student dropped prior to the census date.
For both testing and preparing for the implementation of this SAP customization, it is critical that colleges understand the Statuses and Actions Rules and review/revise their local configurations as needed.
User Acceptance Testing (UAT) for Pass 2 for Breaks in Enrollment & Summer Drops
Session Date/Time: Thursday, May 31, 2023 from 10am to 12pm
UAT Testing Period: Thursday, May 31, 2023 through Friday, June 23, 2023 at 5:00 pm
UAT Sign-Off: June 23, 2023 (No later than 6:30 pm)
UAT Sign-Off Survey: The link to the survey will be distributed at the appropriate time to the test participants. (ONE survey response per college please.)
WebEx UAT Session Recording: https://sbctc.webex.com/webappng/sites/sbctc/recording/d35e2e46e202103ba7fb0050568127c3/playback
Slide Deck Presentation:
Access Information for the UAT Test Environment
Link to PQA Environment: https://pt-pqa.peoplesoft-nonprod-aws.ctclink.sbctc.edu/ptpqa/signon.html
Last Refresh Date: 5/2/2023
Environment Hours: Monday through Friday, 7am to 7pm
Username: Your EMPLID
Password: Will be shared with testers after the session
Questions?
Send an email to Amanda Hoover ([email protected]) and CC Ana Ybarra ([email protected]).
Trouble Logging In?
Send an email to Caitlin Stein ([email protected]).
Need to Report an Issue?
To report an issue discovered during testing please submit a ticket to the Service Desk and provide as many details as possible, including screenshots, IDs used, and steps taken to produce the issue or error.
Service Desk Ticketing System: https://servicedesk.sbctc.edu/
Request Type: ctcLink Support > Testing
Subject: UAT – FA Pass 2 & Summer Drops – [please add your short description of the issue]
Statuses and Actions Rules
For both testing and preparing for the implementation of this SAP customization, it is critical that colleges understand the Statuses and Actions Rules and review/revise their local configurations as needed.
The Statuses and Actions functionality compares the previous SAP term status with the currently Calculated SAP Status to determine the Final SAP Status.
- Previous SAP Status: Select the most recently calculated term or aid year status value to be evaluated.
- Calc SAP Status: Select the value of the current calculated SAP status.
- Final SAP Status: Select the Overall SAP Status Code for term to be assigned if a student's Previous SAP Status and Calc SAP Status match the values selected in this row.
Image: Example of a local Statuses and Actions Rules configuration
Navigation: Set Up SACR > Product Related > Financial Aid > Satisfactory Academic Progress > Define Set Up Criteria >> Criteria 1 tab
Example: If a student’s prior Calc SAP Status was PROB and the current term evaluation would result in a WARN status, the test would evaluate the Statuses and Actions Rules and would result in a SUSP status, based on the example setup PROB + WARN = SUSP.
How the “Calc SAP Status” is Determined
Each SAP Status code has a Severity level, per the college’s local configuration. After the SAP process has evaluated each test, the program stores the resulting SAP Status for each of those tests under the “SAP Calculation Results” section of the Student SAP page. To the right of each Status is the Severity level. The SAP Status with the highest Severity value will be the “Current SAP Status” for the SAP term being evaluated. This is then used in the Statuses and Actions Rules, if they are being used, to determine the final “Calc SAP Status” on the Student’s SAP page. If the student does not match a Statuses and Actions Rule, then the "Current SAP Status" value will be entered as the "Calc SAP Status" on the Student SAP page.
Image: Student SAP page indicating the severity level of the SAP Statuses calculated for each SAP test
Navigation: Financial Aid > Satisfactory Academic Progress > Maintain Student SAP Data
Pass 1 and Pass 2 Processes
The Define Evaluation Terms local configuration defines term to term processing. If the student has enrollment and grade statistics within the Evaluation Term defined for the Term to be processed, the calculation will be considered for Pass 1 Processing. Otherwise, the calculation will use the most recent term with statistics and will be evaluated for Pass 2 Processing.
Note for Fall Term Only: Fall Term is the only row with two Evaluation Terms listed (Spring and Summer), as Summer Term is optional for attendance. Enrollment in either of these two defined terms will result in a Pass 1 evaluation.
The Student SAP Page will indicate in the Processing Message whether the SAP Status was determined via Pass 1 or Pass 2 processing, as well as from which enrollment term it pulled statistics to determine the SAP Status for each of the 8 SAP tests.
Image: Setup terms used to evaluate Satisfactory Academic Progress
Navigation: Set Up SACR > Product Related > Financial Aid > Satisfactory Academic Progress > Define Evaluation Terms
Image: Evaluation Terms setup for Fall
Navigation: Set Up SACR > Product Related > Financial Aid > Satisfactory Academic Progress > Define Evaluation Terms
Image: Student SAP Page will indicate in the Processing Message
Navigation: Financial Aid > Satisfactory Academic Progress > Maintain Student SAP Data
In the visual below are four examples of how the FA SAP processing logic is returning an incorrect SAP Status for students who have a break in enrollment.
Example Student "Tan New"
The Summer 2021 SAP Status before the fix was being returned as "WARN", rather than "SUSP" due to the program simply returning the Calculated SAP Status, rather than performing the comparison to the last valid SAP status, where a "WARN" SAP Status and a "WARN" SAP status should have resulted in a suspension (SUSP), rather than simply returning the student back to warning once again.
Example Student "Margo New"
The Summer 2021 SAP Status before the fix was being returned as "WARN", rather than "SUSP" due to the program simply returning the Calculated SAP Status, rather than performing the comparison to the last valid SAP status, where a "PROB" SAP Status and a "WARN" SAP status should have resulted in a suspension (SUSP), rather than simply returning the student back to warning once again.
Example Student "Kelly Conversion"
The Summer 2021 SAP Status before the fix was being returned as "WARN", rather than "SUSP" due to the program simply returning the Calculated SAP Status, rather than performing the comparison to the last valid SAP status, where a "WARN" SAP Status and a "WARN" SAP status should have resulted in a suspension (SUSP), rather than simply returning the student back to warning once again.
Example Student "Ari Conversion"
The Fall 2021 SAP Status before the fix was being returned as "WARN", rather than "2SUS" due to the program simply returning the Calculated SAP Status, rather than performing the comparison to the last valid SAP status, where a "SUSP" SAP Status and a "WARN" SAP status should have resulted in a suspension (2SUS), rather than simply returning the student back to warning once again.
The visual below highlights the issue encountered by FA Offices prior to the fix. In the sample data you can see that SAP was run for Winter 2020 (2201) and evaluated the progress made in Fall 2019 (2197). The highest priority Calculated SAP Status for that term was WARN, but since this was a Pass 1 (no break in enrollment) the SAP program referenced the prior term attended that also resulted in a WARN status. The program then used both WARN statuses in the Statuses and Actions rule of WARN + WARN = SUSP, resulting in the final SAP Status of a Suspension.
Image: Example of student's previous term SAP page
Navigation: Financial Aid > Satisfactory Academic Progress > Maintain Student SAP Data
The next term that SAP ran for was Spring 2020 (2203). The student did not enroll in Winter 2020 (2201), so the FA SAP Process went back and again evaluated that last term where the student was actively enrolled: Fall 2019 (2197). That term again resulted in a Calculated SAP Status of WARN. Since there was a break in enrollment, the Pass 2 logic was engaged and no evaluation of the Student SAP Status for a prior term was performed. The Final SAP Status was set to WARN. This forced the Financial Aid Office to manually override the Student SAP Status from WARN to SUSP.
Image: Example of Student's current term SAP page that calculated incorrectly due to current problematic behavior in production environment
Navigation: Financial Aid > Satisfactory Academic Progress > Maintain Student SAP Data
Pass 1 Current System Behavior (No Change):
- When executed, the SAP program will first perform a Pass 1 evaluation for all students who were enrolled in the SAP Evaluation Term, and the Fully Graded Date for that term has passed.
- If more than one Evaluation Term is configured, then the program will refer to the Final SAP Status on the more recent (or “Max”) Evaluation term.
- The Calculated SAP Status is determined by running all applicable SAP tests, per the college’s local SAP configuration, then selecting the SAP Status with the highest Severity Level.
- The Statuses and Actions Rules are then used to compare the Calculated SAP Status to the student’s Previous SAP Status to determine the Final SAP Status for the term.
- *See Financial Aid: SAP Local Configuration Guide for proper local configuration on Define SAP Setup pages for Statuses and Actions and proper term setup on the Define Evaluation Terms page. In order for SAP to process, the system (delivered) requires an ‘Active’ FA Term Activation page must exist for each student intending to be processed for SAP.
Pass 2 Current System Behavior (Prior to Change):
- If no data exists for Pass 1, student had a break in enrollment, the SAP processing logic will perform Pass 2.
- Pass 2 SAP Status is determined by running all applicable SAP tests and selecting the SAP Status with the highest Severity Level.
- Statuses and Actions Rules are not used in Pass 2; the Calculated SAP Status will not be compared to a Previous SAP Status.
- On the Run Control for the Process SAP option:
- If the “Update using Most Recent Evaluation Terms (Pass 2)” is selected, then when the program runs Pass 2 it will update the Student SAP page with the results of Pass 2.
- Otherwise, the program will not update that student’s SAP Status, as Pass 1 results did not exist and Pass 2 will not be posted.
To address the issues raised by colleges a modification to existing SAP processing logic was needed to ensure that student SAP is accurately calculated, not just on those students who continuously enroll [Current Functionality], but also for those students who have a break in enrollment [New Functionality].
The following changes have been made to the SAP Processing Logic for all SAP Tests:
- Perform the Previous SAP Status and Calculated SAP Status comparison using the Statuses and Actions local configuration options on Pass 2 (students with a break in enrollment activity).
- Modify the program so when the SAP Process reads the SAP Status from the Maintain Student SAP Data page for a previous term, it compares that term to the enrollment table and will only utilize the Student SAP Status as the Previous SAP Status for comparison if that term has enrollment data.
Solution Design Document Attachment (Approved by FA Community on 5/26/2023)
Steps in the Test Process
Test Scope
In Scope for Testing:
- Students who have had a failure to meet SAP in their academic history in ctcLink, have a break in enrollment and would trigger a different SAP Status from their Calculated SAP Status if a comparison to the Previous SAP Status was taken into account.
- Students who have had a failure to meet SAP in their academic history in ctcLink, have a break in enrollment and would trigger a different SAP Status from their Calculated SAP Status if a comparison to the Previous SAP Status was taken into account and had a record on the Maintain Student SAP Data page for a term they subsequently performed a complete withdrawal from.
Out of Scope for Testing:
- Colleges with no Statuses and Actions Rules configured and/or “Use Statuses and Actions” box on setup is not checked
- Scenarios that will always require human intervention, e.g., appeals and overriding a SAP status to PLAN
- Students who do not have a SAP Status calculated for their previous term of enrollment
- Students who don’t match the Statuses and Actions Rules
- Recalculating SAP on a term that has an Override Status on the most recent row for that term
Selecting Your Student Populations for Testing
- Identify a list of at least 5-10 students who have breaks in enrollment in their academic history. Since manually rebuilt transcript students are considered 'out of scope' for this test activity, focus on Financial Aid students who have breaks in enrollment, or complete withdrawals for a term after a SAP Status was calculated during an enrollment period when the college was on ctcLink or were part of your converted student population.
- Test for other data conditions beyond just those impacted by the bug itself to ensure that the program changes did not adversely effect other students' SAP Test results.
- Review the Recommended Test Scenarios section of this guide to ensure there is at least 1 student (although more than 1 is recommended) per scenario.
- If your college does not have a student that meets a particular criteria you can work with the FA Support Team to determine if another college is covering this scenario or determine how to stage a student that meets that criteria.
If the student's SAP won't calculate, you may need to check the student's FA Term and make sure that it has an Active row (QRG: 9.2 Building an FA Term).
SAP will also not use Statuses and Actions if the previous term of enrollment does not have a calculated SAP Status
Finding Students via Query
The query QCS_FA_SAP_PASS2_SUMMERDROPS has been developed in PQA for testing purposes. It is designed to find students with no enrollment in the term specified who fit into the specified testing scenario.
Prompts:
- Institution - Institution code
- Term w/ No Enrl - Term with a SAP Status and no enrollment that should be excluded from Statuses and Actions with this new fix
- Calculated SAP Term - Current term SAP being calculated
- Prev SAP Status - Status of previous term with enrollment ("Prev SAP Status" from Statuses and Actions Rules)
- Current SAP (Optional) - SAP Status for current term ("Calc SAP Status" from Statuses and Actions Rules)
Image: Example of query prompts and results to find students with a break in enrollment (non Summer Drop)
Navigation: Reporting Tools > Query > Query Viewer
Image: Example of query prompts and results to find students for Summer Drops issue
Navigation: Reporting Tools > Query > Query Viewer
Example Data Scenarios
- Failed to meet SAP after grading occurred for Winter 2021 Term, which placed the Student SAP Status for Spring 2021 at “WARN.”
- Failed to meet SAP again after grading occurred for Spring 2021 Term, which placed the Student SAP Status for Summer 2021 at “SUSP.”
- Student enrolled in Summer 2021 but withdrew from all classes before the start of the term.
- BEFORE FIX: When SAP would have been run for Fall 2021 it would determine that this student had a ‘break in enrollment’ and therefore would skip the Statuses and Actions Rules evaluation, thus the student would have a Final SAP Status of “WARN.”
-
AFTER FIX: When SAP runs for Fall 2021 it will determine that this student had a ‘break in enrollment’ and therefore the new logic would read this as a Pass 2 and will perform the Statuses and Actions Rules evaluation as follows:
- The program will skip the Student SAP Status of “SUSP” from Summer 2021 since there was no enrollment record for Summer 2021.
- The Statuses and Actions Rules evaluation would instead read the Student SAP Status of “WARN” from Spring 2021 since that is the term with the last enrollment record.
- The program will treat the Student SAP Status of “WARN” from Spring 2021 as the “Previous” SAP Status and compare it to the Calculated SAP Status of “WARN.”
- Thus “WARN” plus “WARN”, as seen in the sample rules below, will result in a Final Calc Status of “SUSP”.
- Failed to meet SAP after grading occurred for Fall 2021 Term and resulted in a Student SAP Status of SUSP. Student appealed suspension and was placed on Probation for Winter 2021 and manually assigned an Override Status of PROB.
- Failed to meet cumulative GPA so would have failed SAP again after grading occurred for Winter 2021 Term, however the student did not enroll for Spring and SAP was not run.
- BEFORE FIX: When SAP would have been run for Summer 2021 it would be determined that this student had a ‘break in enrollment’ and therefore the program would skip the Statuses and Actions Rules evaluation, thus the student would have a Final SAP Status of WARN for Summer 2021.
-
AFTER FIX: When SAP runs for Summer 2021 it will determine that this student had a ‘break in enrollment’, read this as a Pass 2, and perform the Statuses and Actions Rules evaluation as follows:
- The program will skip Spring 2021 since there was no enrollment record for Spring 2021.
- The Statuses and Actions Rules evaluation will instead read the overridden Student SAP Status of “PROB” from Winter 2021 as the “Previous” SAP Status and compare it to the Calculated SAP Status of “WARN.”
- Thus “PROB” plus “WARN”, as seen in the sample rules below, will result in a Final Calc Status of “SUSP”.
Recommended Test Scenarios
- To test for the Pass 2 change conditions tester must identify students who have a break in enrollment and fail one or more SAP Tests.
- To test for the Summer Drop condition, tester must identify students who have a Maintain Student SAP Data page for a term where enrollment was dropped prior to the Census Date and has a SAP Status in a Maintain Student SAP Data page for a prior term where they had enrollment past the Census Date.
How to Verify Your Test Outcomes
Success Examples
| Test/Scenario | Expected Outcome |
|---|---|
| Matches a Statuses & Actions Rule, and the previous SAP does not contain an Override Status |
|
Matches a Statuses & Actions Rule, and the previous SAP contains an Override Status. |
|
Does not match a Statuses & Actions Rule, and the previous SAP does not contain an Override Status. |
|
Does not match a Statuses & Actions Rule, and the previous SAP contains an Override Status. |
|
| Add a new row and add a new Statuses & Actions Rule or modify an existing one. |
|
| Batch Process SAP |
|
Failure Examples
- Unexpected error messages in Process Monitor (for the Process SAP batch process)
- Unexpected error message when selecting the Calculate button the Student SAP page
- Pass 2 does not use Statuses and Actions when the student's scenario matches an already defined Statuses and Actions Rule
- Summer Drop issue persists
Troubleshooting
- SAP will not calculate for a term if the corresponding FA Term is Inactive.
- SAP will not use Statuses and Actions if the previous term of enrollment does not have a calculated SAP Status
Local Configuration Guide
Online Materials for Satisfactory Academic Progress Local Configuration :
- Quick Reference Guide(s): CS: Local Configuration Guide - Financial Aid - SAP Configurations
Security Matrix Mapping Information
The information below provides those completing the Security Matrix Mapping templates with the Menus, Pages and Role names needed to execute the User Acceptance Testing relevant to this UAT guide.
Running SAP via Batch Process:
MENU: Main Menu > Financial Aid > Satisfactory Academic Progress > Process SAP
ROLE NAMES: ZZ FA SAP Prcs
Running SAP via Maintain Student SAP Data page:
MENU: Main Menu>Financial Aid>Satisfactory Academic Progress>Maintain Student SAP Data
ROLE NAMES: ZZ FA SAP Prcs
0 Comments
Add your comment