Conformance validations and testing
This section contains descriptions and links to test and validation tools, which are for public use. Testing scenarios, test data, and test results are also listed. Information of a similar type available elsewhere is linked.
Conformity standards
In the EWP network, conformity standards are defined on various levels and with various means:
Formal specification of the EWP network architecture;
Formal specification of EWP network security protocols;
Formal specification of APIs for exchange of mobility data and supporting various mobility scenarios;
Examples of data records, use cases, and test scenarios;
Supporting documents explaining mandatory business requirements in a less formal way;
Validators that, on the one hand, define the data format and format of requests and responses and, on the other, provide tools for automatic validation.
There are rules in place which enforce the compliance with the standards defined for the network:
Entry procedure with mandatory steps
to perform automatic validation tests;
to execute testing with the reference implementation, i.e., the EWP Dashboard;
The validation tests carried out by the EWP technical team as part of the acceptance procedure;
The registration process is when the mobility software is finally approved for production use.
The conformance validation is facilitated by the DEMO environment, which can be used by the EWP technical team, but also the mobility software providers upon request. The Dashboard development team maintains test installations used for testing with the reference implementation.
There are various meeting places and forums where providers can discuss technical issues. They also play an important role in clarifying standards and facilitating the development of standards-compliant software:
Regular meetings of the technical community of mobility software providers on the Infrastructure Forum, where all technical issues are thoroughly discussed, new APIs introduced, and their specifications elaborated.
Technical workshops for the providers.
Discussions in GitHub issues are available to the public.
EWP network Validation tools
There are three groups of tools for automatic testing of the EWP APIs:
XML Schema Validator
XML Schema Validator helps developers with writing EWP XML documents. It should be possible to validate any XML document described in all RELEASED, DEPRECATED, and OBSOLETE specifications (plus, perhaps, some of the DRAFT ones).
XML Schema Validator can be found on the DEV Registry Service website in the XML Schema Validator section (similarly in the PROD Registry). A developer can copy any produced XML document, including the manifest file, and validate its compliance with the XSD schema.
Documents that do not pass the test should not be sent via the EWP network. Implementations have the right to reject incorrect documents.
API Validator
API Validator is a tool that helps developers determine if their implementation conforms to the EWP specifications. It plays an extremely important role in the development process. The validator is integrated with the EWP registry and available on GitHub. It can be used either in a local environment or online from the DEV Registry Service website.
A developer can install the registry in their local environment and test their development installation, even before contacting the EWP network administrator. There is a package in GitHub that can be downloaded and installed in the local environment. The README-local-registry.md document describes the possible testing options. The simplest option, Scenario 2: Using web validator, is described in it.
If a developer would like to learn about other possibilities of running the Validator, test their installation registered in the development Registry, or test their implementation of another partner in the network, they should read EWP-Validator-HOWTO.txt describing all validation options.
If a developer’s installation is already publicly available, they can use the online Validator, which is located on the Dev Registry Service website in the Manifest Importer Status section, to test public APIs. It doesn't have access to confidential data, so you will not be able to test APIs like omobilities or LAs.
IIA Hash Validator
This validator checks the hash of the cooperation conditions present in the IIA to get response.
IIA Hash Validator can be found on the DEV Registry Service website in the IIA Hash Validator section (similarly in the PROD Registry). A developer can copy any <iia> element obtained from the IIA get response, and validate if the <conditions-hash> element contains a proper value.
The source code in Java used for hash calculation and validation is posted in the GitHub repository. It can be used by Java developers in their solution. Other developers can grasp the idea by reading the code.
DEMO Testbed
The API validators can be used for testing responses to the requests, but can not be used for testing the client side, in particular scenarios which require human intervention. This can be done with the EWP DEMO testbed instead.
The DEMO testbed is a set of online services. Test installations of a student information system with the EWP connector and the mobility client, with the most recent versions of IIA and LA APIs, have been set up at the University of Warsaw.
The DEMO testbed is only available in the EWP DEV network. This testbed contains the following applications:
DEMO adm – system for administration of a receiving institution (SCHAC=demo.usos.edu.pl),
HEI adm – system for administration of a sending institution (SCHAC=hei.demo.usos.edu.pl),
USOSweb – portal for students and mobility coordinators with the mobility module where LA for outgoing students is created.
IRK – admission system for applicants and mobility coordinators where LA for incoming students is received.
All these installations contain test data (anonymised). More data can be entered via the web interface.
These applications are used by the team from the University of Warsaw for testing EWP scenarios with other providers. Write to ewp-tech@lists.erasmuswithoutpaper.eu to schedule the test session.
Teams that would like to test by themselves can get credentials upon request.
There is a separate document, DEMO Testbed. User Guide, which explains how the testbed operates how to prepare test data and run tests. The current version of the user guide describes inter-institutional agreements (part I) and learning agreements (part II), but new chapters will be added with descriptions of other processes, like nominations and Transcript of Records.
Having access to their own installation and the DEMO testbed installation, a developer can test full scenarios, both on the server and client sides, which could not be possible with the validators. It is recommended to run scenarios stored in GitHub repositories for particular APIs.
It is also possible to use DEMO applications for exporting sample data to build unit tests of developers’ own software. LAs can be easily exported from the admission system in a raw XML format. Transcripts of Records can be exported from the student’s portal (in raw XML and in PDF).
Test scenarios
GitHub repositories contain not only specifications of the APIs, but also sample data and test scenarios:
They are written in the Gherkin notation. They are used by the EWP team for acceptance testing with the reference implementation. They may also be used by any development team for bilateral testing, inside local systems with their own installations, in the DEV network with any partner, or from within the DEMO testbed.
Obligations of providers before joining the EWP network
The procedure of joining the EWP network for both groups of providers is described in detail in the Joining the EWP network section of the IT Service Catalogue. In particular:
The provider MUST install and test its mobility software in the DEV network.
The provider MUST perform automated self-tests with the validators.
The provider MUST test with the reference implementation. The EWP technical team will go through the testing scenarios as described in GitHub.
After successful testing, the provider may enter the production network. Only APIs that were tested can be included in the manifest file. When the provider would like to declare endpoints for new processes, they need to undergo the testing with the reference implementation.