Generate tests

You can use Kolo to generate an integration test from a trace.

Warning

Generating tests is in Alpha. Expect bugs and please report them on GitHub or via email to w@kolo.app. Please also get in touch if our documentation doesn’t give you the information you need to use these features.

How to generate a test from a trace with Kolo

Run pip install -U "kolo[test_generation]"
To generate a test from a trace, we need to know the trace id and the file where we’d like to put the test.

The command to use is kolo generate-test [trace_id] > [filepath]
So for example: kolo generate-test trc_01GZZWS4D4TA8PVJ9D040KEKZ3 > main/my_test.py

To get the trace_id, you can right click on one of the traces in the VSCode sidebar and copy it to your clipboard:

Or alternatively you can run kolo trace list to see the trace ids for your most recent traces.
Finally, run your freshly generated test, for example using python manage.py main.my_test 🎉

Customising test generation

Kolo’s test generation makes several assumptions about the structure of your test. You can customise the name of the generated test class and test method:

kolo generate-test trc_01GZZWS4D4TA8PVJ9D040KEKZ3 --test-class="GeneratedTestCase" --test-name="test_generated"

Kolo also supports generating a pytest-style test by specifying the django_request_pytest.py.j2 template:

kolo generate-test trc_01GZZWS4D4TA8PVJ9D040KEKZ3 --template=django_request_pytest.py.j2

Django Model Factories

If you use factory boy you can tell Kolo where to find your factories:

# .kolo/config.toml

[test_generation]

factories = [
    {"path": "tests.factories.UserFactory"},
    {"path": "tests.factories.ArticleFactory", "pk": true},
    ...
]

path should be a dotted path to the factory class. pk tells Kolo to include the recorded primary key value in the factory create call.

Templates

If you want more control, you can define your own jinja template:

kolo generate-test trc_01GZZWS4D4TA8PVJ9D040KEKZ3 --template="path/to/template.py.j2"

For example, this cut-down version of Kolo’s pytest template:

{% if sql_fixtures or asserts or test_client_call_pytest %}
import pytest

{% for import in imports %}{{ import }}
{% endfor %}

@pytest.mark.django_db()
def {{ test_name }}(client):
    {% for fixture in sql_fixtures %}
    {% for line in fixture.template_parts %}
    {{ line }}
    {% endfor %}
    {% endfor %}

    {% if test_client_call_pytest %}
    response = {{ test_client_call_pytest }}

    assert response.status_code == {{ response.status_code }}

    {% for fixture in asserts %}
    {% for line in fixture.pytest_template_parts %}
    {{ line }}
    {% endfor %}
    {% endfor %}
{% endif %}

Trace processors

For complete flexibility, you can define custom “trace processors”. These are functions that take a context and return data that will be merged into the context. For example:

def my_function_processor(context):
    frames = context["_frames"]
    my_function_frames = [
        f for f in frames if f["type"] == "frame" and f["co_name"] == "my_function"
    ]
    call_args = [f["locals"] for f in my_function_frames if f["event"] == "call"]
    return_values = [f["arg"] for f in my_function_frames if f["event"] == "return"]
    
    return {"my_function_calls": list(zip(call_args, return_values))}

Your custom processors should be added to your .kolo/config.toml file:

# .kolo/config.toml

[test_generation]

trace_processors = [
    "path.to.my_function_processor",
]

You can now use my_function_calls in a custom template, or in another processor later in the list.

Field parsers

Kolo supports many model fields provided by Django. Sometimes you may have a third-party field which isn’t being parsed correctly. In this case you can define a custom field parser:

def parse_custom_field(value, field):
    if field != "dotted.field.path.CustomField":
        return value

    # custom parsing logic

    return parsed_value

# .kolo/config.toml

[test_generation]

field_parsers = [
    "path.to.parse_custom_field",
]

Known limitations

Kolo test generation is very early stage, so expect errors both in the test generation and in the ultimate test execution. As you encounter these, please let us know so that we can look into them. :heart_eyes:
We try to auto generate test fixtures for you, but in some cases we don’t have enough data to generate them. In that case, you will need to either add additional fixtures manually or tweak the ones we generate.
Time machine or freezegun is used to make sure dealing with time is easier and more consistent. If you’re not using one of these, you will need to install one or define a custom template or processor for your preferred library.
httpretty is used to mock expected outbound http requests. If you’re not using httpretty, you will need to install it or manually change httpretty to your preferred http mocking library.