MATLAB certification/verification/validation
21 views (last 30 days)
Show older comments
I am trying to find what certification is done on MATLAB. There must be some ISO, ASTM, IEEE standard it is tested to.
How do I know (trust) MATLAB is doing things correctly?
I have tried searching.....I have tried seearch "MATLAB certification" and this gets me how a user can credentials. I have searched "MATLAB verification and validation" and this gets me how to write a veriffcation and validatuon script. I have also searched the MATLAB documentation.
2 Comments
dpb
on 12 May 2022
As a wholly-owned proprietary product, whatever testing is done on MATLAB is entirely within TMW's internal QA/QC program which they do not publicize. There is no Standards group for MATLAB as there is for the major compiled languages. Although the base product is compiled with such toolset(s), much of the distributed functionality is in the proprietary m-files or other compiled p-code if not compiled completely to native code.
The most that there is you can rely on is that the base math operations are built from such compiler tools with their compliance to the appicable Standard (although surprisingly little of even those Standards has to do with accuracy or correctness of results; they deal mostly with the required semantics and behavior of the compiler and compliance of syntax, etc.) and the references to the available external libraries used like FFTW, LINPACK, etc., for other areas.
If one is in an environment that requires some sort of certification of toolsets, about the only way to do so with MATLAB would be to have an internal test suite against which to compare results. There's at least one frequent contributor here who is in such an environment and does have such a process in place; consequently his organization is relatively slow in accepting new releases. He probably will see this and have more in-depth comments on specifics.
In former life where was involved in licensing and safety-related calculations for power reactors under the NRC and US Code 10CFR40, all calculational tools we used for such calculations had to undergo a verification and acceptance process; MATLAB was not an accepted toolset for such reasons that it would have required another independent verification of any models/calculations done using it to have them accepted for licensing/safety calculations
The vendor- and commercially-supplied Fortran compilers were accepted/used to build the verfied/accepted calculational models referenced above -- any compiler updates required a revisiting of a test suite of those codes to verify the produced the same results when rebuilt with a new compiler version--the compilers themselves were not the verified tool. That would be the same process with MATLAB -- it would be the model that would be verified, not MATLAB itself and reverification of the model with updates to MATLAB. Given the frequency with which TMW releases updates, that could be a sizable burden on an organization to keep up with.
Answers (2)
Michelle Hirsch
on 12 May 2022
I assume that what you are getting at is ultimately trying to assess quality. While standards have their place, it's not obvious to me that compliance with any given standard assures you that everything is actually correct.
I would look more at processes. We run something like 250,000 unique automated tests on our algorithms. When I checked on specifics years ago, I heard that we run over 2 million test points a day. Additionally, MathWorks follows standard software development best practices. The most important one from my perspective here is our "fix as you go" approach which is meant to stop defects from being passed along in the development process. Development teams need to keep their bug counts low, and can't work on new features if they aren't meeting bug count targets.
None of these things "guarantee" correctness, but we've learned over the years that rigorous testing combined with processes that minimize code changes close to release are very effective techniques for driving the high quality for which MATLAB is known.
If you have specific certification requirements or questions, don't hesitate to reach out to tech support. They can provide more details relevant to your specific needs than I can do in this forum.
2 Comments
Steven Lord
on 13 May 2022
That figure of 250,000 unique automated tests that Michelle cited is correct: that's about how many test files we have. But to clarify that's not the number of test cases since each test file can contain many test cases. As one example the test file for the plus operator contains thousands of test cases.
See Also
Categories
Find more on Software Development Tools in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!