PropWise - the library to detect and suggest code deserving property based testing

Another outcome of Language Agnostic Code Audit SaaS I’m currently working on is a PropWise library.

It scans the project content, and does its best on detecting and reporting places suitable for property-based testing, with examples using StreamData or PropEr.

An excerpt from docs:

PropWise analyzes your Elixir code to find functions that would benefit from property-based testing. It examines the Abstract Syntax Tree (AST) of your code to:

▸ Detect pure functions (functions without side effects)
▸ Identify common patterns suitable for property testing
▸ Find inverse function pairs (encode/decode, serialize/deserialize, etc.)
▸ Score and rank candidates by testability
▸ Provide specific testing suggestions for each candidate

The example of ouroborosing itself:

❯ mix propwise
Compiling 8 files (.ex)
Generated propwise app
Analyzing ....

================================================================================
PropWise Analysis Report
================================================================================

Summary:
  Total functions analyzed: 115
  Property test candidates: 30
  Candidates dropped (below threshold): 76
  Coverage: 26.1%

--------------------------------------------------------------------------------
Top Candidates (sorted by score):
--------------------------------------------------------------------------------

PropWise.Reporter.format_json_report/1
  Score: 10
  Location: lib/prop_wise/reporter.ex:133
  Type: private
  Patterns:
    - Numeric Algorithm: Arithmetic operations
    - Data Transformation: Map transformation
    - Collection Operation: Uses Enum collection operations
  Testing suggestions:
    - property "returns numeric result" do
        check all n <- one_of([integer(), float()]) do
          result = Reporter.format_json_report(n)
          assert is_number(result)
        end
      end

    - property "handles zero and negative inputs" do
        check all n <- one_of([integer(), float()]) do
          # Verify the function doesn't crash on edge-case numeric inputs.
          _ = Reporter.format_json_report(n)
        end
      end

…

Enjoy!

7 Likes

Huh? What does that mean exactly?

Also awesome library. Will try that plus Telemetria soon.

I’m working on SaaS application providing the language-agnostic software audit. Three libraries I posted this week (this one, Muex, and OeditusCredo) are the part of this effort working with BEAM languages only. Proof of concept, to be precise.

2 Likes