Why and When

Happy users and happy programmers is an obvious, but also very important, goal for us all. Therefore, we have set up smoke test guidelines for all extension developers and we strongly recommend that everyone run smoke test before each major release (May & November)and minor release (February & August).

This document describes why, and gives guidelines on when the smoke tests preferably should be executed. Our hope with introducing the smoke test guidelines is to:

• Get more robust Extension and a more robust Core.
• Find issues in time.
• Get happier user.
• Get happier programmers.
• Have a smoother and less stressful releases for everyone: Core, Extension, developers and users.
• Improve Extension and Core quality and usability.

If you have any questions, please don't hesitate to contact our Developer Support at devsupport@configura.com.

Why?

Running smoke tests is great for you and the users in many different ways. Please note that smoke testing is not the same as a full regression test. A full regression test is also recommended before each major release.

The smoke test helps you:

• Detect new issues in your own Extension in good time before the release. i.e. you have time to fix any problems.
• Find issues in related to Core functionality early enough for the Core team to resolve them in time for the upcoming release.
• Find quality improvements and new quality issues. Which includes user-friendliness, reliability and usability issues.
• The full regression test is done after the smoke test. Thanks to the smoke test, you will likely save time during the regression test as the easy to spot problems will not slow things down. In general, you save time and money in total when a smoke test has been performed.

When?

Developer can start migration approximately 7 weeks before major release. It is recommended that you run the migration testing and smoke testing once the developer has done the migration. The testing should be repeated with the Release Candidate. For minor release, although partners might not need to do the migration, it is recommended that you run the smoke testing using the Release Candidate which is available 4 weeks before the release. Please complete all testing at least 2 weeks before the release. This is to allow the core developers to resolve the issue before release.

Guidelines

This page provide a list of test where we strongly recommended that you perform before each CET Designer major and minor release. Please note that these are guidelines for smoke test, not a regression test.

Automated testing

We recommend that you run automatic tests on your extension. To help with this CET Developer provides some of different automatic test functionalities that you can employ:

Use the following automation tools found in QA Tools extension:

• PDF Diff
• Order Export Diff
• Batch Load

Insert All Snappers

• Use the Insert from Current Toolbox button in the Release Debug Dialog (Ctrl+Alt+12)
• Make sure the snappers are visible in 2D and 3D without any crash.

Manual Testing

The idea behind the manual testing is to simulate a real use case, by drawing the same scene for each release.

We recommend that you record the test (for example using a tool called Jing). This makes it easier to repeat the exact same test for upcoming major releases.

By doing this you will:

• Detect new bugs and crashes.
• Find quality improvements and new quality issues:
  • Is it easy to draw your test case drawing?
  • Has anything changed? (Better or worse than previous release?)

Real user scenario smoke test guidelines

1. Check these after installing CET with your extensions.
   • Catalogues download automatically.
   If your customers are always installing your extensions with other extensions, please make sure your extensions are compatible with those extensions too. If you have any question regarding testing, please post your question in the Quality Assurance Forum.

2. Create a basic manufacturer drawing by using these functionalities:

   • Connect several symbols to each other to form a group
   • Animation: placing, stretching, snapping, rotate, moving etc.
   • Single select, Multi select, Rectangle select, Lasso select.
   • Copy, paste and cut
   • Create blocks and Component Blocks (if applicable)
   • Create Alternatives
   • Use replicate tool (Polar, Linear and Rectangle.)
   • Create Multi floor. Check to place Symbols on different floors.
   • Turn on multiple drawing function. Check that you can work with multiple drawings at once.
   • Attach dimension tools to symbols.
   • Enter project information.

3. Once the drawing is completed, perform these tests:

   • Change the quick properties and animation properties for a few symbols
   • Check that the 2D and 3D graphics are correct (E.g. length, color, shape, etc.)
   • Undo and Redo
   • Change material on some symbols using Material Explorer
   • Change material on some symbols using Schemes
   • Freeze and unfreeze some symbols
   • Group and ungroup some symbols
   • Remove some symbols from the drawing
   • Create and load favorites
   • Tag a few symbols
   • Lock and unlock symbols.
   • Check that the Categories are present and correct
   • Check that prices are visible in Calculation dialog
   • Use order export functionality in the Calculation dialog. Do a quick check for making sure the export works fine
   • Render your drawing, both through Quick Render and from within the Photo Lab
   • Generate Elevations (E.g. panel elevation, basic elevation, etc.)
   • Create a picklist, save, load, edit, save and load again
   • Switch between view modes
   • Switch between different Filter and Visibility settings
   • Paper view: create 2D and 3D viewports and make sure your symbols are visible in the viewports
   • Generate auto papers
   • Print Preview, Print and create PDF
   • Save the drawing, load, edit and then save and load again
   • Export drawing
   • Check that the extension’s icons, labels and tooltips still look correct
     Switch to different languages if the extension supports multiple languages
   • Check that the performance is not worse than the previous version
     Use the Response Monitor and Model Performance tool to help with testing

4. Test loading old test data created from previous CET Designer versions

   • Drawings (.cmdrw)
   • Favorite files (.cmfav)
   • Paper (.cmppr)

Migration Testing

Please run the migration testing by referring to the latest Migration Testing Guide.