This site has been put in static mode and is no longer active. You will not be able to login, but public content
will be visible for reference purposes. Please visit our new site at http://www.fixtradingcommunity.org.

Implementation Guide

Contents | Previous | Next | Terms of Use

5. Testing a FIX implementation

Why trading partners need to test

The FIX Protocol is the global standard for the electronic exchange of trading information, and much of its success can be attributed to its flexibility and openness. While the FPL publishes a specification inclusive of accepted FIX messages, structure, message flows and required versus optional tags, trading applications often behave in different ways depending on the functionality offered and business needs they satisfy. Different trading applications may require a specific tag or value within a message that the FPL has listed as optional or perhaps not defined at all. In addition to the published tags, FIX also reserves a large suite of tags that can be 'user defined', providing an additional layer of flexibility and opportunity for customization. This allows varying trading applications and systems to customize and differentiate themselves, offering different functionality for different business purposes. This openness and flexibility means that system compatibility must be ensured via a comprehensive testing process.

Testing compatibility with trading partners before "going live" is a critical step in the connectivity process. Since all products and environments have their own unique features, there are many areas where potential problems can arise. The purpose of testing is to identify and fix these problems before they have a chance to negatively impact something in production.

What to test

Certification is about ensuring that your trading partners' systems are compatible with yours. The interfaces you've chosen (e.g. FIX versions, message types), the way you represent information (e.g. product identifiers, currencies), and the types of transactions you support are all components of this interface and must all be tested thoroughly with each trading partner. Therefore, during the testing process, it is recommended that each possible scenario be tested both on a session and an application level.

While the session level is relatively consistent across implementations within a version of FIX, the application level allows for much flexibility regarding the different types of messages supported as well as the information communicated in them. Because of this testing each scenario a firm has incorporated within their applications is necessary to ensure the highest degree of compatibility.

Testing Scope & Effort

The following give a guide as to the breakdown in resources for each testing cycle.

Connectivity Testing10%
FIX testing (session)10%
FIX/OMS (front office)40%
Integration testing, E-2-E40%

Sample testing scripts

FIX session level testing is fairly standard in nature within each version. Application testing will depend mostly on the types of trading a firm does and the capabilities of the order entry application. Testing script should be inclusive of all scenarios a firm expects its traders and order entry application(s) to encounter, and inclusive of all expected and required behaviors of a firm's trading partners. Sample application testing scripts have been included in Appendix A at the end of this section.

The Global Fixed Income Committee has begun the process of creating a suite of test scripts specific to fixed income products. The availability of fixed income scripts will allow application developers to have an industry "certified" standard with which to test their message flow and required/optional tags. The work is results from collaboration between the Technical and the Business Practice Subcommittees. The work will follow the successful method of documenting business practices through extensive industry input and then a template will be given to the Technical Committee to generate scripts. See the Fixed Income Section of www.fixprotocol.org for updates. The Global Fixed Income Committee hopes to publish these scripts in the fourth quarter of 2003.

Buy side and Sell side approach

Buyside and sellside firms that have built their own trading applications should seek to test and certify with all of their trading partners. It is recommended that each firm establish and document a "Rules of Engagement" guide with regard to FIX implementations. Generally speaking, the buyside and sellside should agree on many different parameters regarding their interface, including but not limited to:
  • Network connectivity and support
  • Infrastructure (e.g., internal LAN, Citrix, Desktop setup) and compatibility with other applications (Bloomberg, Reuters, etc.)
  • FIX version
  • Client vs. Server relationship
  • FIX header message information
  • All FIX message types supported
  • All required tags within each FIX message type and status.
  • While the FIX Protocol defines required versus optional tags within each message, different firm's applications often require tags that the FPL did not require.

Vendor approach

One of the main selling points for vendor-based FIX solutions is that the client who purchases the vendor application can reap the benefits of previous testing and certification between the vendor and various trading partners. While buy side and sell side firms seek to test and certify their applications with their respective trading partners, vendor systems generally seek to test and certify with as large a number of trading partners as possible. While client demand will ultimately drive the priority of who tests and when, it is in the vendor's best interest to offer "off the shelf" connectivity to as many pools of liquidity as possible. This will drastically reduce the amount of time it takes the buyer of such a system to begin trading with their trading partners if they have already certified with the vendor in question.

With regard to FIX testing and certification, there are generally two approaches vendors can take also. They can either maintain all update capabilities to the software themselves, or allow the client to make certain changes to the source code. Allowing clients to access source code does provide greater flexibility, but reduces the level of service and support the vendor can provide. Prohibiting access to source code makes it easier to provide better service and support, but the user loses flexibility in terms of changing how the interface behaves. Therefore, depending on the vendor software a firm chooses to purchase, the FIX testing and certification process may be outsourced to the vendor. If the client is allowed access to source code, they would need to be involved in the testing and certification process as well if they choose to make changes to it.

Vendor software is generally designed for either buy side or sell side clients. Depending on this, each vendor tests and certifies with the appropriate trading partners. While buy side and sellside firms that build their own trading software only focus on testing with their specific trading partners, vendors generally seek to certify with as many trading partners as possible.

With regard to the set-up of FIX sessions with trading partners for vendors, there are two basic models followed:

  • Service Bureau - The vendor maintains one FIX session with a firm, and manages each client's order flow via that session, identifying trading partners using the appropriate FIX tags.
  • Point-to-Point - The vendor maintains multiple sessions with each firm, establishing one per client or perhaps many. For example, different trading desks or business units within a firm may require/desire separate FIX sessions.

Choosing one of the above methods is a matter of preference. There is nothing that can be communicated via the FIX message using a service bureau set-up that cannot be sent via a point-to-point connection.

Issues with testing/what can go wrong

Historically, testing has been a manual process, where a representative from each firm connects a development system with his/her trading partner's development system and they run through the various scenarios in each others testing scripts. This approach has generally proven successful if a firm needs to connect with a small number of trading partners and has very limited trading functionality. However, the proliferation of ECNs and ATSs, the increased globalization of the marketplace, as well as more sophisticated trading tools and functionality have created many issues with regards to FIX testing and certification. Testing in the past has required trading partners to schedule a mutually acceptable time for them to test. If a firm has numerous trading partners to test with and limited testing resources, the scheduling process can often get difficult and very busy. Once a date and time are agreed upon, each set of test servers and applications must be functioning in order to test, which is not always the case with development/test/QA systems.

While one of the greatest strengths of the FIX Protocol is its flexibility, it does allow for different interpretations and implementations (discussed in the first section regarding 'Why the Need to Test'). This means that one or both of the trading partners will inevitably have to make changes to their engine and/or application during the testing process in order to certify. In some instances these are simple changes that can be made during the scheduled testing period, but often they are larger changes that require users to change their code (or in the case of systems they have bought from vendors, contact their vendor to make the changes). In this case, testing must be stopped and re-scheduled for a later date. Depending on the scale of the change(s), this can add significant length to the testing process.

The aforementioned scenarios address efficiency issues associated with the testing process. There are also potential issues surrounding the quality and effectiveness of testing. Test/development/QA machines often lose access to log files over time, which makes troubleshooting problems extremely difficult, especially if there are long gaps of time in between testing sessions. Adding to the frustration level is the quality of the tests performed. When dealing with manual processes, human error is always a concern. Ensuring that instructions regarding a system's behavior, and that all testing scenarios have been successfully completed accurately is paramount and difficult to guarantee.

Automated Testing

Certification is about ensuring that your trading partners systems are compatible with yours. The interfaces you've chosen (e.g. FIX versions, message types), the way to represent information (e.g. product identifiers, currencies), and the types of transactions you support are all components of this interface and must all be tested thoroughly with each trading partner.

With appropriate technology, certification can be automated. This creates a couple of advantages. First, automating tests ensures that nothing gets missed during the certification process. Not only does it make sure that every scenario a firm requires its trading partners to test is completed, but it also makes certain that the components of the FIX message are in accordance with the design of a firm's system. Additionally, by logging the results of each test, it creates transparency with regards to the structure of the FIX messages and helps identify areas where a firm may be struggling against your system's implementation. Second, automates testing means that people are no longer required to manually participate in every single test session. This creates much greater flexibility in terms of scheduling and systems availability, and allows many trading partners to test simultaneously.

Automated certification requires a technology platform that is flexible enough to simulate any conceivable scenario that might be necessary in your environment.

Questions to ask during and after testing

  1. Do I have the best combination of information to ensure a maximum of successful trade messages (i.e. adding in Tag 15:Currency, Tag 100:Exchange destination)
  2. Symbology database: "is it clean & up to date?"
  3. Standardization: where is the industry moving to with symbology?
    • RIC's are most intuitive symbology but intellectual property of Reuters
    • ISIN's are most widely accepted (outside of the US)
    • No real consensus at present but a very important issue going forward.

How can you argue for sufficient testing resources?

Business Drivers - Maximize ROI by:

  1. Making sure the implementation is effective and efficient the first time will reduce the probability of having to do it again in the future. This greatly reduces operational as well as opportunity costs.
  2. Reducing processing errors from the front office trade capture through to allocation & settlement errors (better DMA capabilities).
  3. Effective and efficient testing methods will increase access to more trading partners in a shorter amount of time, helping to grow the business.

Technology Drivers - Implement best practices to:

  1. Compress delivery cycles & do it right the first time!
  2. Adhere to budgetary constraints.
  3. Reduce potential processing errors.

Summary of Key points

  1. Plan, document, and gain the resources you need to run an effective test program.
  2. Identify the areas that need testing, then develop the scenarios and scripts to exercise them.
  3. Implement the FIX solution that best matches your firms and your OMS's capabilities.

Contents | Previous | Next | Terms of Use