Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trying to fix-Init should refuse if there are existing results #560

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion src/cosmic_ray/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,13 @@ def new_config(config_file):


@cli.command()
@click.option("--force", is_flag=True, default=False, help="Force initialization")
@click.argument("config_file")
@click.argument(
"session_file",
# help="The filename for the database in which the work order will be stored."
)
def init(config_file, session_file):
def init(config_file, session_file,force):
"""Initialize a mutation testing session from a configuration. This
primarily creates a session - a database of "work to be done" -
which describes all of the mutations and test runs that need to be
Expand Down Expand Up @@ -96,7 +97,13 @@ def init(config_file, session_file):
log.info(" - %s: %s", directory, ", ".join(sorted(files)))

with use_db(session_file) as database:

if database.num_results>0 or not force:
return


cosmic_ray.commands.init(modules, database, operators_cfg)


sys.exit(ExitCode.OK)

Expand Down
4 changes: 3 additions & 1 deletion tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

import pytest


@pytest.fixture
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixture doesn't seem to be doing anything. It should be removed.

def force():
return "True"
@pytest.fixture
def tmpdir_path(tmpdir):
"""A temporary directory as a pathlib.Path."""
Expand Down
54 changes: 53 additions & 1 deletion tests/unittests/test_command_line_processing.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,16 @@
import stat

import pytest

from exit_codes import ExitCode

import cosmic_ray.cli
import cosmic_ray.config
import cosmic_ray.modules
import cosmic_ray.mutating
import cosmic_ray.plugins

from cosmic_ray.work_db import WorkDB
from cosmic_ray.work_item import WorkItem,MutationSpec

@pytest.fixture
def config_file(tmpdir):
Expand Down Expand Up @@ -64,6 +66,56 @@ def test_non_existent_config_file_returns_EX_NOINPUT(session, local_unittest_con
assert cosmic_ray.cli.main(["exec", "no-such-file", str(session)]) == ExitCode.CONFIG



def test_init_with_existing_results_no_force(session, local_unittest_config):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll probably need to use the lobotomize fixture here to prevent CR from trying to actually scan the non-existent module references in local_unittest_config.

"""Test that init exits without reinitializing when results exist and force=False"""
# Sample WorkItem creation
mutation_spec = MutationSpec(
module_path="src/example/test.py",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pass a Path object here, not a string. I know that MutationSpec will convert it, but I'd like to get away from depending on that.

operator_name="delete_line",
occurrence=1,
start_pos=(10, 0),
end_pos=(10, 20),
operator_args={"comment": "Delete print statement"}
)

work_items = [
WorkItem(
job_id="test_job_123",
mutations=(mutation_spec,)
)
]

db=WorkDB(session,1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use WorkDB.Mode.create here instead of just 1.

Also, it would be more idiomatic to use the use_db context manager rather than opening the WorkDB directly like this.

db.add_work_items(work_items)

# Verify initial database state has our test work items
initial_count = db.num_work_items
assert initial_count == len(work_items)



result = cosmic_ray.cli.main(["init", local_unittest_config, str(session)])
db=WorkDB(session,2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use WorkDB.Mode.open here, not just 2. And use use_db.

final_count = db.num_results
assert final_count == initial_count # Confirm work items are unchanged
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You seem to be comparing the wrong things here. final_count is the number of results, i.e. the number of work items which have actually been executed. initial_count is the simply the total number of work items, whether or not they have results attached to them. You need to actually get some results into the workdb for this test to make any sense, and even then you need to make sure you compare the number of results before and after your init call.

assert result == ExitCode.OK










def test_init_with_existing_results_force(session, local_unittest_config,force):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned earlier, you don't need (and aren't really using) the force fixture here. Remove it.

"""Test that reinitialization occurs when force=True"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't testing what it claims to test. You're not re-initializing the database, just initializing it, so passing --force doesn't do anything special in this case. You need to:

  1. Create the WorkDB
  2. Put some results into it and verify that it has the results
  3. Re-initialize it with --force
  4. Assert that it now contains no results.


result = cosmic_ray.cli.main(["init", local_unittest_config, str(session),"--force"])
assert result == ExitCode.OK

@pytest.mark.skip("need to sort this API out")
def test_unreadable_file_returns_EX_PERM(tmpdir, local_unittest_config):
p = tmpdir.ensure("bogus.session.sqlite")
Expand Down