issues with dpseg/AG installations still there?

[alecristia]

A moment ago, I cloned yet another copy of wordseg, followed instructions for system-wide installation. Yet the install doesn't fully pass tests:
Alejandrinas-MacBook-Air:build acristia$ make test
Running tests...
Test project /Users/acristia/Documents/wordseg_utest/build
Start 1: wordseg.test_algo_ag.py
1/16 Test #1: wordseg.test_algo_ag.py ............***Failed 2.39 sec
Start 2: wordseg.test_algo_baseline.py
2/16 Test #2: wordseg.test_algo_baseline.py ...... Passed 0.58 sec
Start 3: wordseg.test_algo_dibs.py
3/16 Test #3: wordseg.test_algo_dibs.py .......... Passed 1.56 sec
Start 4: wordseg.test_algo_dpseg.py
4/16 Test #4: wordseg.test_algo_dpseg.py .........***Failed 14.74 sec
Start 5: wordseg.test_algo_puddle.py
5/16 Test #5: wordseg.test_algo_puddle.py ........ Passed 2.91 sec
Start 6: wordseg.test_algo_tp.py
6/16 Test #6: wordseg.test_algo_tp.py ............ Passed 0.64 sec
Start 7: wordseg.test_evaluate.py
7/16 Test #7: wordseg.test_evaluate.py ........... Passed 0.63 sec
Start 8: wordseg.test_folding.py
8/16 Test #8: wordseg.test_folding.py ............ Passed 1.46 sec
Start 9: wordseg.test_pipeline.py
9/16 Test #9: wordseg.test_pipeline.py ...........***Failed 5.85 sec
Start 10: wordseg.test_prepare.py
10/16 Test #10: wordseg.test_prepare.py ............ Passed 0.78 sec
Start 11: wordseg.test_separator.py
11/16 Test #11: wordseg.test_separator.py .......... Passed 0.94 sec
Start 12: wordseg.test_statistics.py
12/16 Test #12: wordseg.test_statistics.py ......... Passed 0.75 sec
Start 13: wordseg.test_syllabification.py
13/16 Test #13: wordseg.test_syllabification.py .... Passed 0.65 sec
Start 14: wordseg.test_tutorial.py
14/16 Test #14: wordseg.test_tutorial.py ...........***Failed 13.05 sec
Start 15: wordseg.test_utils.py
15/16 Test #15: wordseg.test_utils.py .............. Passed 0.70 sec
Start 16: wordseg.test_wordseg_commands.py
16/16 Test #16: wordseg.test_wordseg_commands.py ... Passed 7.52 sec

75% tests passed, 4 tests failed out of 16

Total Test time (real) = 55.19 sec

The following tests FAILED:
1 - wordseg.test_algo_ag.py (Failed)
4 - wordseg.test_algo_dpseg.py (Failed)
9 - wordseg.test_pipeline.py (Failed)
14 - wordseg.test_tutorial.py (Failed)
Errors while running CTest
make: *** [test] Error 8

Note that no errors during brew:
$ brew install python3 boost cmake
Warning: python3 3.6.4_2 is already installed
Warning: boost 1.66.0 is already installed
Warning: cmake 3.10.1 is already installed

[alecristia]

Apologies, I should have included the output of pytest -v
============================= test session starts ==============================
platform darwin -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /anaconda/bin/python
cachedir: ../.cache
rootdir: /Users/acristia/Documents/wordseg_utest, inifile:
collecting ... collected 195 items

../test/test_algo_ag.py::test_grammar_files PASSED
../test/test_algo_ag.py::test_ag_single_run FAILED
../test/test_algo_ag.py::test_most_common_parse PASSED
../test/test_algo_ag.py::test_yield_trees PASSED
../test/test_algo_ag.py::test_setup_seed PASSED
../test/test_algo_ag.py::test_grammars[Coll3syllfnc_enFestival.lt-Word] FAILED
../test/test_algo_ag.py::test_grammars[Colloc0_enFestival.lt-Colloc0] FAILED
../test/test_algo_ag.py::test_mark_jonhson FAILED
../test/test_algo_ag.py::test_eval FAILED
../test/test_algo_baseline.py::test_proba_bad[1.01] PASSED
../test/test_algo_baseline.py::test_proba_bad[-1] PASSED
../test/test_algo_baseline.py::test_proba_bad[a] PASSED
../test/test_algo_baseline.py::test_proba_bad[True] PASSED
../test/test_algo_baseline.py::test_proba_bad[1] PASSED
../test/test_algo_baseline.py::test_hello PASSED
../test/test_algo_dibs.py::test_basic[baseline-0-None] PASSED
../test/test_algo_dibs.py::test_basic[baseline-0-0.2] PASSED
../test/test_algo_dibs.py::test_basic[baseline-0.5-None] PASSED
../test/test_algo_dibs.py::test_basic[baseline-0.5-0.2] PASSED
../test/test_algo_dibs.py::test_basic[baseline-1-None] PASSED
../test/test_algo_dibs.py::test_basic[baseline-1-0.2] PASSED
../test/test_algo_dibs.py::test_basic[phrasal-0-None] PASSED
../test/test_algo_dibs.py::test_basic[phrasal-0-0.2] PASSED
../test/test_algo_dibs.py::test_basic[phrasal-0.5-None] PASSED
../test/test_algo_dibs.py::test_basic[phrasal-0.5-0.2] PASSED
../test/test_algo_dibs.py::test_basic[phrasal-1-None] PASSED
../test/test_algo_dibs.py::test_basic[phrasal-1-0.2] PASSED
../test/test_algo_dibs.py::test_basic[lexical-0-None] PASSED
../test/test_algo_dibs.py::test_basic[lexical-0-0.2] PASSED
../test/test_algo_dibs.py::test_basic[lexical-0.5-None] PASSED
../test/test_algo_dibs.py::test_basic[lexical-0.5-0.2] PASSED
../test/test_algo_dibs.py::test_basic[lexical-1-None] PASSED
../test/test_algo_dibs.py::test_basic[lexical-1-0.2] PASSED
../test/test_algo_dibs.py::test_phone_sep PASSED
../test/test_algo_dibs.py::test_replicate_cdswordseg PASSED
../test/test_algo_dpseg.py::test_dpseg_parallel_folds[1-1] FAILED
../test/test_algo_dpseg.py::test_dpseg_parallel_folds[1-4] FAILED
../test/test_algo_dpseg.py::test_dpseg_parallel_folds[2-2] FAILED
../test/test_algo_dpseg.py::test_dpseg_parallel_folds[10-1] PASSED
../test/test_algo_dpseg.py::test_dpseg_args[] FAILED
../test/test_algo_dpseg.py::test_dpseg_args[--ngram 1 --a1 0 --b1 1 --a2 0 -- b2 1] FAILED
../test/test_algo_dpseg.py::test_dpseg_args[--ngram 1 --a1 0.1 --b1 0.9 --do-mbdp 1] FAILED
../test/test_algo_dpseg.py::test_dpseg_args[--ngram 1 --a1 0 --b1 1 --forget-method P] FAILED
../test/test_algo_dpseg.py::test_dpseg_args[--ngram 1 --a1 0 --b1 1 --estimator V --mode batch] FAILED
../test/test_algo_dpseg.py::test_dpseg_args[--ngram 1 --a1 0 --b1 1 --estimator F] FAILED
../test/test_algo_dpseg.py::test_dpseg_args[--ngram 1 --a1 0 --b1 1 --estimator T] FAILED
../test/test_algo_dpseg.py::test_dpseg_args[--ngram 1 --a1 0 --b1 1 --estimator D --mode online --eval-maximize 1 --eval-interval 50 --decay-rate 1.5 --samples-per-utt 20 --hypersamp-ratio 0 --anneal-a 10 --burnin-iterations 1 --anneal-iterations 0] FAILED
../test/test_algo_dpseg.py::test_config_files_are_here PASSED
../test/test_algo_dpseg.py::test_dpseg_from_config_file[/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/bi_dmcmc.conf] FAILED
../test/test_algo_dpseg.py::test_dpseg_from_config_file[/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/dps.conf] FAILED
../test/test_algo_dpseg.py::test_dpseg_from_config_file[/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/viterbi.conf] FAILED
../test/test_algo_puddle.py::test_puddle[1-1-3] PASSED
../test/test_algo_puddle.py::test_puddle[1-1-10] PASSED
../test/test_algo_puddle.py::test_puddle[1-5-3] PASSED
../test/test_algo_puddle.py::test_puddle[1-5-10] PASSED
../test/test_algo_puddle.py::test_puddle[5-1-3] PASSED
../test/test_algo_puddle.py::test_puddle[5-1-10] PASSED
../test/test_algo_puddle.py::test_puddle[5-5-3] PASSED
../test/test_algo_puddle.py::test_puddle[5-5-10] PASSED
../test/test_algo_tp.py::test_tp[relative-forward] PASSED
../test/test_algo_tp.py::test_tp[relative-backward] PASSED
../test/test_algo_tp.py::test_tp[absolute-forward] PASSED
../test/test_algo_tp.py::test_tp[absolute-backward] PASSED
../test/test_algo_tp.py::test_hello_world PASSED
../test/test_evaluate.py::test_read_data PASSED
../test/test_evaluate.py::test_gold_on_gold PASSED
../test/test_evaluate.py::test_ipa PASSED
../test/test_evaluate.py::test_basic_1 PASSED
../test/test_evaluate.py::test_basic_2 PASSED
../test/test_evaluate.py::test_basic_3 PASSED
../test/test_evaluate.py::test_basic_4 PASSED
../test/test_evaluate.py::test_basic_5 PASSED
../test/test_folding.py::test_permute PASSED
../test/test_folding.py::test_flatten PASSED
../test/test_folding.py::test_boundaries PASSED
../test/test_folding.py::test_fold PASSED
../test/test_folding.py::test_unfold_basic[1] PASSED
../test/test_folding.py::test_unfold_basic[2] PASSED
../test/test_folding.py::test_unfold_basic[3] PASSED
../test/test_folding.py::test_fold_unfold_nfolds[1] PASSED
../test/test_folding.py::test_fold_unfold_nfolds[2] PASSED
../test/test_folding.py::test_fold_unfold_nfolds[4] PASSED
../test/test_folding.py::test_fold_unfold_nfolds[6] PASSED
../test/test_folding.py::test_fold_unfold_nfolds[8] PASSED
../test/test_folding.py::test_fold_unfold_nfolds[10] PASSED
../test/test_pipeline.py::test_pipeline[baseline-ascii] PASSED
../test/test_pipeline.py::test_pipeline[baseline-unicode] PASSED
../test/test_pipeline.py::test_pipeline[ag-ascii] FAILED
../test/test_pipeline.py::test_pipeline[ag-unicode] FAILED
../test/test_pipeline.py::test_pipeline[puddle-ascii] PASSED
../test/test_pipeline.py::test_pipeline[puddle-unicode] PASSED
../test/test_pipeline.py::test_pipeline[tp-ascii] PASSED
../test/test_pipeline.py::test_pipeline[tp-unicode] PASSED
../test/test_pipeline.py::test_pipeline[dibs-ascii] PASSED
../test/test_pipeline.py::test_pipeline[dibs-unicode] PASSED
../test/test_pipeline.py::test_pipeline[dpseg-ascii] FAILED
../test/test_pipeline.py::test_pipeline[dpseg-unicode] FAILED
../test/test_prepare.py::test_paiwise PASSED
../test/test_prepare.py::test_bad_utterances[] PASSED
../test/test_prepare.py::test_bad_utterances[ ] PASSED
../test/test_prepare.py::test_bad_utterances[

] PASSED
../test/test_prepare.py::test_bad_utterances[hello] PASSED
../test/test_prepare.py::test_bad_utterances[ah ;esyll ah] PASSED
../test/test_prepare.py::test_bad_utterances[ah ;esyll ah ;esyll] PASSED
../test/test_prepare.py::test_bad_utterances[ah ;esyll ah ;eword] PASSED
../test/test_prepare.py::test_bad_utterances[ah ah ; eword] PASSED
../test/test_prepare.py::test_bad_utterances[;eword] PASSED
../test/test_prepare.py::test_bad_utterances[;eword a b ;esyll ;eword] PASSED
../test/test_prepare.py::test_bad_utterances[;esyll a b ;esyll ;eword] PASSED
../test/test_prepare.py::test_bad_utterances[a. ;eword] PASSED
../test/test_prepare.py::test_bad_utterances[a! ;eword a b ;esyll ;eword] PASSED
../test/test_prepare.py::test_good_utterances[a ;eword] PASSED
../test/test_prepare.py::test_good_utterances[a ;esyll ;eword] PASSED
../test/test_prepare.py::test_good_utterances[\xcb\x8c\xca\x8c s ;eword t \xc9\x99 ;eword p l e\xc9\xaa ;eword \xc3\xb0 \xc9\x99 ;eword s o\xca\x8a l o\xca\x8a z ;eword] PASSED
../test/test_prepare.py::test_prepare_text[phone] PASSED
../test/test_prepare.py::test_prepare_text[syllable] PASSED
../test/test_prepare.py::test_prepare_bad_types PASSED
../test/test_prepare.py::test_prepare_tolerant PASSED
../test/test_prepare.py::test_punctuation[n i2 ;eword s. w o5 ;eword s. \xc9\x99\xc9\x9c n ;eword m o-\xc9\x9c ;eword] PASSED
../test/test_prepare.py::test_punctuation[\xc9\x915 ;eword j i\xc9\x9c ;eword (en) a\xc9\x9c m (zh) ;eword] PASSED
../test/test_prepare.py::test_punctuation[t u b l i ;eword p o i i s^ s^ ;eword] PASSED
../test/test_separator.py::test_levels PASSED
../test/test_separator.py::test_iterate PASSED
../test/test_separator.py::test_bad_separators PASSED
../test/test_separator.py::test_good_separators PASSED
../test/test_separator.py::test_strip PASSED
../test/test_separator.py::test_remove PASSED
../test/test_separator.py::test_remove_level PASSED
../test/test_separator.py::test_remove_bad PASSED
../test/test_separator.py::test_remove_re PASSED
../test/test_separator.py::test_split_vs_tokenize[..p.s.p.-expected0-True] PASSED
../test/test_separator.py::test_split_vs_tokenize[..p.s.p.-expected1-False] PASSED
../test/test_separator.py::test_split_vs_tokenize[..p.s.p.w-expected2-True] PASSED
../test/test_separator.py::test_split_vs_tokenize[..p.s.p.w-expected3-False] PASSED
../test/test_separator.py::test_split_vs_tokenize[..p.s.p.w..p.s.w-expected4-False] PASSED
../test/test_separator.py::test_split_vs_tokenize[..p.s.p.w..p.s.-expected5-False] PASSED
../test/test_separator.py::test_split_vs_tokenize[..p.s.p.w..p.s.-expected6-True] PASSED
../test/test_separator.py::test_tokenize_2 PASSED
../test/test_separator.py::test_tokenize_noboundaries PASSED
../test/test_separator.py::test_tokenize_full_nosyll PASSED
../test/test_separator.py::test_tokenize_full_syll PASSED
../test/test_separator.py::test_upper_levels PASSED
../test/test_statistics.py::test_most_common PASSED
../test/test_statistics.py::test_unigram PASSED
../test/test_statistics.py::test_descibe1 PASSED
../test/test_statistics.py::test_descibe2 PASSED
../test/test_statistics.py::test_describe3 PASSED
../test/test_statistics.py::test_entropy PASSED
../test/test_syllabification.py::test_bad_input PASSED
../test/test_syllabification.py::test_remove_phones PASSED
../test/test_syllabification.py::test_remove_restore_phones[] PASSED
../test/test_syllabification.py::test_remove_restore_phones[ab c] PASSED
../test/test_syllabification.py::test_remove_restore_phones[a;b;] PASSED
../test/test_syllabification.py::test_remove_restore_phones[a;b; ] PASSED
../test/test_syllabification.py::test_remove_restore_phones[a;b; c;] PASSED
../test/test_syllabification.py::test_remove_restore_phones[n;o; s;e; k;a;e; ] PASSED
../test/test_syllabification.py::test_cspanish PASSED
../test/test_syllabification.py::test_cspanish_bad PASSED
../test/test_syllabification.py::test_cspanish_strip PASSED
../test/test_syllabification.py::test_cspanish_phones PASSED
../test/test_syllabification.py::test_cspanish_phones_strip PASSED
../test/test_tutorial.py::test_tutorial[py] FAILED
../test/test_tutorial.py::test_tutorial[sh] PASSED
../test/test_utils.py::test_counting_iterator_int[sequence0] PASSED
../test/test_utils.py::test_counting_iterator_int[sequence1] PASSED
../test/test_utils.py::test_counting_iterator_int[sequence2] PASSED
../test/test_utils.py::test_counting_iterator_int[sequence3] PASSED
../test/test_utils.py::test_counting_iterator_int[sequence4] PASSED
../test/test_utils.py::test_counting_iterator_str PASSED
../test/test_utils.py::test_counting_iterator_bad[1] PASSED
../test/test_utils.py::test_counting_iterator_bad[None] PASSED
../test/test_utils.py::test_get_binary_good[ag] PASSED
../test/test_utils.py::test_get_binary_good[dpseg] PASSED
../test/test_utils.py::test_get_binary_bad[AG] PASSED
../test/test_utils.py::test_get_binary_bad[Dpseg] PASSED
../test/test_utils.py::test_get_binary_bad[puddle] PASSED
../test/test_utils.py::test_get_config_file_good[ag] PASSED
../test/test_utils.py::test_get_config_file_good[dpseg] PASSED
../test/test_utils.py::test_get_config_file_bad[puddle] PASSED
../test/test_utils.py::test_get_config_file_bad[tp] PASSED
../test/test_utils.py::test_get_config_file_bad[dibs] PASSED
../test/test_utils.py::test_strip_abc[ ab c ] PASSED
../test/test_utils.py::test_strip_abc[ab
c] PASSED
../test/test_utils.py::test_strip_abc[ ab
c ] PASSED
../test/test_utils.py::test_strip_misc PASSED
../test/test_wordseg_commands.py::test_command_help[prep] PASSED
../test/test_wordseg_commands.py::test_command_help[eval] PASSED
../test/test_wordseg_commands.py::test_command_help[stats] PASSED
../test/test_wordseg_commands.py::test_command_help[syll] PASSED
../test/test_wordseg_commands.py::test_command_help[baseline] PASSED
../test/test_wordseg_commands.py::test_command_help[ag] PASSED
../test/test_wordseg_commands.py::test_command_help[dibs] PASSED
../test/test_wordseg_commands.py::test_command_help[dpseg] PASSED
../test/test_wordseg_commands.py::test_command_help[tp] PASSED
../test/test_wordseg_commands.py::test_command_help[puddle] PASSED

=================================== FAILURES ===================================
______________________________ test_ag_single_run ______________________________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]

def test_ag_single_run(prep):
    raw_parses = ag._run_ag_single(

      prep, os.path.join(grammar_dir, grammars[1][0]), args=test_arguments)

../test/test_algo_ag.py:31:

---

text = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
grammar_file = '/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/ag/Colloc0_enFestival.lt'
args = '-E -P -R -1 -n 10 -a 0.0001 -b 10000.0 -e 1.0 -f 1.0 -g 100.0 -h 0.01 -x 2'
test_text = 'hh ih r\ndh eh r w iy g ow\ns ow ax m p ey sh ax n t\nw ah t d ih d y uw iy t ah p\nuw p s\nl ih t ax l b ih t m ao r...iy b ih t iy b ih t iy b ih t iy b ey b iy g er l\ny uw hh aa s p ae g hh eh t iy ow z\nw aa n t s ax m m ao r\nn ow\n'
log_level = 40, log_name = 'wordseg-ag'

def _run_ag_single(text, grammar_file, args, test_text=None,
                   log_level=logging.ERROR, log_name='wordseg-ag'):
    """Runs the AG program a single time and returns the computed parse trees

    Parameters
    ----------
    text : sequence
        The list of utterances to train the model on, and to segment
        if `test_text` is None.
    grammar_file : str
        The path to the grammar file to use for segmentation
    args : str
        Command line options to run the AG program with, use
        'wordseg-ag --help' to have a complete list of available
        options
    test_text : sequence, optional
        If not None, the list of utterances to segment on the model
        learned from `text`
    log : logging.Logger, optional
        A logger where to send log messages

    Returns
    -------
    A list of parse trees, one tree per input utterance in `text`

    Raises
    ------
    RuntimeError
        If the AG program fails and returns an error code

    """
    log = utils.get_logger(name=log_name, level=log_level)

    # setup the train text as a multiline string
    train_text = '\n'.join(utt.strip() for utt in text) + '\n'

    # setup the test text as well
    test_text = train_text if test_text is None else (
        '\n'.join(utt.strip() for utt in test_text) + '\n')

    # we need to write the test text as a tempfile, so we create a
    # tempdir. The directory and its content is automatically erased
    # when done
    temp_dir = tempfile.mkdtemp()
    try:
        # write the test text as a temporary file. ylt extension is
        # the one used in the original AG implementation
        test_tmpfile = os.path.join(temp_dir, 'test.ylt')
        codecs.open(test_tmpfile, 'w', encoding='utf8').write(test_text)

        # generate the command to run as a subprocess
        command = ('{binary} {grammar} {args} -u {test} '.format(
            binary=utils.get_binary('ag'),
            grammar=grammar_file,
            args=args,
            test=test_tmpfile))

        log.info('running %s', command)

        # run the command as a subprocess
        process = subprocess.Popen(
            shlex.split(command),
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        parses, messages = process.communicate(train_text.encode('utf8'))

        # log.debug the AG messages AFTER EXECUTION
        messages = messages.decode('utf8').split('\n')
        for msg in messages:
            log.debug(re.sub('^# ', '', msg).strip())

        # fail if AG returns an error code
        if process.returncode:
            raise RuntimeError(

              'fails with error code {}'.format(process.returncode))

E RuntimeError: fails with error code -6

../wordseg/algos/ag.py:447: RuntimeError
________________ test_grammars[Coll3syllfnc_enFestival.lt-Word] ________________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
grammar = '/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/ag/Coll3syllfnc_enFestival.lt'
level = 'Word'

@pytest.mark.parametrize('grammar, level', grammars)
def test_grammars(prep, grammar, level):
    grammar = os.path.join(grammar_dir, grammar)

  segmented = ag.segment(prep, grammar, level, test_arguments, nruns=1)

../test/test_algo_ag.py:72:

---

../wordseg/algos/ag.py:719: in segment
for n in range(nruns))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
grammar_file = '/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/ag/Coll3syllfnc_enFestival.lt'
args = '-E -P -R -1 -n 10 -a 0.0001 -b 10000.0 -e 1.0 -f 1.0 -g 100.0 -h 0.01 -x 2 -r 22389'
test_text = 'hh ih r\ndh eh r w iy g ow\ns ow ax m p ey sh ax n t\nw ah t d ih d y uw iy t ah p\nuw p s\nl ih t ax l b ih t m ao r...iy b ih t iy b ih t iy b ih t iy b ey b iy g er l\ny uw hh aa s p ae g hh eh t iy ow z\nw aa n t s ax m m ao r\nn ow\n'
log_level = 30, log_name = 'wordseg-ag - run 1'

def _run_ag_single(text, grammar_file, args, test_text=None,
                   log_level=logging.ERROR, log_name='wordseg-ag'):
    """Runs the AG program a single time and returns the computed parse trees

    Parameters
    ----------
    text : sequence
        The list of utterances to train the model on, and to segment
        if `test_text` is None.
    grammar_file : str
        The path to the grammar file to use for segmentation
    args : str
        Command line options to run the AG program with, use
        'wordseg-ag --help' to have a complete list of available
        options
    test_text : sequence, optional
        If not None, the list of utterances to segment on the model
        learned from `text`
    log : logging.Logger, optional
        A logger where to send log messages

    Returns
    -------
    A list of parse trees, one tree per input utterance in `text`

    Raises
    ------
    RuntimeError
        If the AG program fails and returns an error code

    """
    log = utils.get_logger(name=log_name, level=log_level)

    # setup the train text as a multiline string
    train_text = '\n'.join(utt.strip() for utt in text) + '\n'

    # setup the test text as well
    test_text = train_text if test_text is None else (
        '\n'.join(utt.strip() for utt in test_text) + '\n')

    # we need to write the test text as a tempfile, so we create a
    # tempdir. The directory and its content is automatically erased
    # when done
    temp_dir = tempfile.mkdtemp()
    try:
        # write the test text as a temporary file. ylt extension is
        # the one used in the original AG implementation
        test_tmpfile = os.path.join(temp_dir, 'test.ylt')
        codecs.open(test_tmpfile, 'w', encoding='utf8').write(test_text)

        # generate the command to run as a subprocess
        command = ('{binary} {grammar} {args} -u {test} '.format(
            binary=utils.get_binary('ag'),
            grammar=grammar_file,
            args=args,
            test=test_tmpfile))

        log.info('running %s', command)

        # run the command as a subprocess
        process = subprocess.Popen(
            shlex.split(command),
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        parses, messages = process.communicate(train_text.encode('utf8'))

        # log.debug the AG messages AFTER EXECUTION
        messages = messages.decode('utf8').split('\n')
        for msg in messages:
            log.debug(re.sub('^# ', '', msg).strip())

        # fail if AG returns an error code
        if process.returncode:
            raise RuntimeError(

              'fails with error code {}'.format(process.returncode))

E RuntimeError: fails with error code -6

../wordseg/algos/ag.py:447: RuntimeError
_________________ test_grammars[Colloc0_enFestival.lt-Colloc0] _________________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
grammar = '/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/ag/Colloc0_enFestival.lt'
level = 'Colloc0'

@pytest.mark.parametrize('grammar, level', grammars)
def test_grammars(prep, grammar, level):
    grammar = os.path.join(grammar_dir, grammar)

  segmented = ag.segment(prep, grammar, level, test_arguments, nruns=1)

../test/test_algo_ag.py:72:

---

../wordseg/algos/ag.py:719: in segment
for n in range(nruns))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
grammar_file = '/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/ag/Colloc0_enFestival.lt'
args = '-E -P -R -1 -n 10 -a 0.0001 -b 10000.0 -e 1.0 -f 1.0 -g 100.0 -h 0.01 -x 2 -r 31666'
test_text = 'hh ih r\ndh eh r w iy g ow\ns ow ax m p ey sh ax n t\nw ah t d ih d y uw iy t ah p\nuw p s\nl ih t ax l b ih t m ao r...iy b ih t iy b ih t iy b ih t iy b ey b iy g er l\ny uw hh aa s p ae g hh eh t iy ow z\nw aa n t s ax m m ao r\nn ow\n'
log_level = 30, log_name = 'wordseg-ag - run 1'

def _run_ag_single(text, grammar_file, args, test_text=None,
                   log_level=logging.ERROR, log_name='wordseg-ag'):
    """Runs the AG program a single time and returns the computed parse trees

    Parameters
    ----------
    text : sequence
        The list of utterances to train the model on, and to segment
        if `test_text` is None.
    grammar_file : str
        The path to the grammar file to use for segmentation
    args : str
        Command line options to run the AG program with, use
        'wordseg-ag --help' to have a complete list of available
        options
    test_text : sequence, optional
        If not None, the list of utterances to segment on the model
        learned from `text`
    log : logging.Logger, optional
        A logger where to send log messages

    Returns
    -------
    A list of parse trees, one tree per input utterance in `text`

    Raises
    ------
    RuntimeError
        If the AG program fails and returns an error code

    """
    log = utils.get_logger(name=log_name, level=log_level)

    # setup the train text as a multiline string
    train_text = '\n'.join(utt.strip() for utt in text) + '\n'

    # setup the test text as well
    test_text = train_text if test_text is None else (
        '\n'.join(utt.strip() for utt in test_text) + '\n')

    # we need to write the test text as a tempfile, so we create a
    # tempdir. The directory and its content is automatically erased
    # when done
    temp_dir = tempfile.mkdtemp()
    try:
        # write the test text as a temporary file. ylt extension is
        # the one used in the original AG implementation
        test_tmpfile = os.path.join(temp_dir, 'test.ylt')
        codecs.open(test_tmpfile, 'w', encoding='utf8').write(test_text)

        # generate the command to run as a subprocess
        command = ('{binary} {grammar} {args} -u {test} '.format(
            binary=utils.get_binary('ag'),
            grammar=grammar_file,
            args=args,
            test=test_tmpfile))

        log.info('running %s', command)

        # run the command as a subprocess
        process = subprocess.Popen(
            shlex.split(command),
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        parses, messages = process.communicate(train_text.encode('utf8'))

        # log.debug the AG messages AFTER EXECUTION
        messages = messages.decode('utf8').split('\n')
        for msg in messages:
            log.debug(re.sub('^# ', '', msg).strip())

        # fail if AG returns an error code
        if process.returncode:
            raise RuntimeError(

              'fails with error code {}'.format(process.returncode))

E RuntimeError: fails with error code -6

../wordseg/algos/ag.py:447: RuntimeError
______________________________ test_mark_jonhson _______________________________

tmpdir = local('/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_mark_jonhson0')
datadir = '/Users/acristia/Documents/wordseg_utest/test/data'

def test_mark_jonhson(tmpdir, datadir):
    # this is a transcription of the original "toy run" delivered with
    # the original AG code (as a target in the Makefile)
    assert os.path.isdir(datadir)

    grammar_file = os.path.join(datadir, 'ag_testengger.lt')
    text = codecs.open(
        os.path.join(datadir, 'ag_testeng.yld'), 'r', encoding='utf8')
    arguments = (
        '-r 1234 -P -D -R -1 -d 100 -a 1e-2 -b 1 -e 1 -f 1 '
        '-g 1e2 -h 1e-2 -n 10 -C -E -A {prs} -N 10 -F {trace} -G {wlt} '
        # -X "cat > {X1}" -X "cat > {X2}" '
        # -U "cat > {prs1}" -v {testeng2} -V "cat > {prs2}"'
        '-u {testeng1} '.format(
            testeng1=os.path.join(datadir, 'ag_testeng1.yld'),
            testeng2=os.path.join(datadir, 'ag_testeng2.yld'),
            trace=tmpdir.join('trace'), wlt=tmpdir.join('wlt'),
            prs=tmpdir.join('prs')
            # X1=tmpdir.join('X1'), X2=tmpdir.join('X2'),
            # prs1=tmpdir.join('prs1'), prs2=tmpdir.join('prs2')))
        ))

  ag._run_ag_single(text, grammar_file, args=arguments)

../test/test_algo_ag.py:101:

---

text = <open file '/Users/acristia/Documents/wordseg_utest/test/data/ag_testeng.yld', mode 'rb' at 0x10d157270>
grammar_file = '/Users/acristia/Documents/wordseg_utest/test/data/ag_testengger.lt'
args = '-r 1234 -P -D -R -1 -d 100 -a 1e-2 -b 1 -e 1 -f 1 -g 1e2 -h 1e-2 -n 10 -C -E -A /private/var/folders/mx/f_gpzmq105x8s...est-of-acristia/pytest-10/test_mark_jonhson0/wlt -u /Users/acristia/Documents/wordseg_utest/test/data/ag_testeng1.yld '
test_text = 'the dog bites a man
the man bites a dog
a man gives the dog a bone
the dog gives a man the bone
a dog bites a bone
'
log_level = 40, log_name = 'wordseg-ag'

def _run_ag_single(text, grammar_file, args, test_text=None,
                   log_level=logging.ERROR, log_name='wordseg-ag'):
    """Runs the AG program a single time and returns the computed parse trees

    Parameters
    ----------
    text : sequence
        The list of utterances to train the model on, and to segment
        if `test_text` is None.
    grammar_file : str
        The path to the grammar file to use for segmentation
    args : str
        Command line options to run the AG program with, use
        'wordseg-ag --help' to have a complete list of available
        options
    test_text : sequence, optional
        If not None, the list of utterances to segment on the model
        learned from `text`
    log : logging.Logger, optional
        A logger where to send log messages

    Returns
    -------
    A list of parse trees, one tree per input utterance in `text`

    Raises
    ------
    RuntimeError
        If the AG program fails and returns an error code

    """
    log = utils.get_logger(name=log_name, level=log_level)

    # setup the train text as a multiline string
    train_text = '\n'.join(utt.strip() for utt in text) + '\n'

    # setup the test text as well
    test_text = train_text if test_text is None else (
        '\n'.join(utt.strip() for utt in test_text) + '\n')

    # we need to write the test text as a tempfile, so we create a
    # tempdir. The directory and its content is automatically erased
    # when done
    temp_dir = tempfile.mkdtemp()
    try:
        # write the test text as a temporary file. ylt extension is
        # the one used in the original AG implementation
        test_tmpfile = os.path.join(temp_dir, 'test.ylt')
        codecs.open(test_tmpfile, 'w', encoding='utf8').write(test_text)

        # generate the command to run as a subprocess
        command = ('{binary} {grammar} {args} -u {test} '.format(
            binary=utils.get_binary('ag'),
            grammar=grammar_file,
            args=args,
            test=test_tmpfile))

        log.info('running %s', command)

        # run the command as a subprocess
        process = subprocess.Popen(
            shlex.split(command),
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        parses, messages = process.communicate(train_text.encode('utf8'))

        # log.debug the AG messages AFTER EXECUTION
        messages = messages.decode('utf8').split('\n')
        for msg in messages:
            log.debug(re.sub('^# ', '', msg).strip())

        # fail if AG returns an error code
        if process.returncode:
            raise RuntimeError(

              'fails with error code {}'.format(process.returncode))

E RuntimeError: fails with error code -6

../wordseg/algos/ag.py:447: RuntimeError
__________________________________ test_eval ___________________________________

datadir = '/Users/acristia/Documents/wordseg_utest/test/data'
tmpdir = local('/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_eval0')

def test_eval(datadir, tmpdir):
    sep = Separator()

    _tags = [utt for utt in codecs.open(
        os.path.join(datadir, 'phonologic.txt'), 'r', encoding='utf8')
            if utt][:200]
    _prepared = list(prepare(_tags, separator=sep))
    _gold = gold(_tags, separator=sep)

    # build a colloc0 grammar from the list of phones
    phones = {p for utt in _prepared for p in utt.split(' ')}
    grammar = ag.build_colloc0_grammar(phones)
    grammar_file = tmpdir.join('grammar.lt')
    grammar_file.write_text(grammar, encoding='utf8')

    segmented = ag.segment(_prepared, str(grammar_file), 'Colloc0',
                           # log=utils.get_logger(level=logging.DEBUG),
                           args=test_arguments.replace('-n 10', '-n 2'),

                         nruns=1)

../test/test_algo_ag.py:122:

---

../wordseg/algos/ag.py:719: in segment
for n in range(nruns))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0283 i\u02d0 d \u028c z \u0250 \u0279 i\u0259 l i w \u028c n d \u025a f \u0259l d\u0292 \u0251\u02d0 b b \u026a k \u028c z \u0283 i\u02d0', 's \u025b n t t \u0259 v i\u02d0 \u0259 t n \u0251\u02d0 m', 'l \u026a \u027e \u0259l b \u026a t t \u028c f \u025a... a\u026a h \u0250 d f \u0279 \u025b n t\u0283', '\xe6 z \u0250 \u0279 \u026a z \u028c l t \xf0 e\u026a t \u028a k \u0279 \u026a l \u026a d\u0292 \u0259 n a\u028a \u027e \u0259 v \xf0 \u0259 p \u0279 o\u028a \u0261 \u0279 \xe6 m \xe6 n d \xf0 e\u026a p \u028a t', ...]
grammar_file = '/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_eval0/grammar.lt'
args = '-E -P -R -1 -n 2 -a 0.0001 -b 10000.0 -e 1.0 -f 1.0 -g 100.0 -h 0.01 -x 2 -r 46338'
test_text = '\u0283 i\u02d0 d \u028c z \u0250 \u0279 i\u0259 l i w \u028c n d \u025a f \u0259l d\u0292 \u0251\u02d0 b b \u026a k \u028c z \u0283 i\u02d0\ns \u025b n t t \u0259 v i\u02d0 \u0259 t n \u0251\u02d0 m\nl \u026a \u027e \u0259l b \u026a t t \u028c f \u025a j u\u02d0 n...z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n\u026a z\n'
log_level = 30, log_name = 'wordseg-ag - run 1'

def _run_ag_single(text, grammar_file, args, test_text=None,
                   log_level=logging.ERROR, log_name='wordseg-ag'):
    """Runs the AG program a single time and returns the computed parse trees

    Parameters
    ----------
    text : sequence
        The list of utterances to train the model on, and to segment
        if `test_text` is None.
    grammar_file : str
        The path to the grammar file to use for segmentation
    args : str
        Command line options to run the AG program with, use
        'wordseg-ag --help' to have a complete list of available
        options
    test_text : sequence, optional
        If not None, the list of utterances to segment on the model
        learned from `text`
    log : logging.Logger, optional
        A logger where to send log messages

    Returns
    -------
    A list of parse trees, one tree per input utterance in `text`

    Raises
    ------
    RuntimeError
        If the AG program fails and returns an error code

    """
    log = utils.get_logger(name=log_name, level=log_level)

    # setup the train text as a multiline string
    train_text = '\n'.join(utt.strip() for utt in text) + '\n'

    # setup the test text as well
    test_text = train_text if test_text is None else (
        '\n'.join(utt.strip() for utt in test_text) + '\n')

    # we need to write the test text as a tempfile, so we create a
    # tempdir. The directory and its content is automatically erased
    # when done
    temp_dir = tempfile.mkdtemp()
    try:
        # write the test text as a temporary file. ylt extension is
        # the one used in the original AG implementation
        test_tmpfile = os.path.join(temp_dir, 'test.ylt')
        codecs.open(test_tmpfile, 'w', encoding='utf8').write(test_text)

        # generate the command to run as a subprocess
        command = ('{binary} {grammar} {args} -u {test} '.format(
            binary=utils.get_binary('ag'),
            grammar=grammar_file,
            args=args,
            test=test_tmpfile))

        log.info('running %s', command)

        # run the command as a subprocess
        process = subprocess.Popen(
            shlex.split(command),
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        parses, messages = process.communicate(train_text.encode('utf8'))

        # log.debug the AG messages AFTER EXECUTION
        messages = messages.decode('utf8').split('\n')
        for msg in messages:
            log.debug(re.sub('^# ', '', msg).strip())

        # fail if AG returns an error code
        if process.returncode:
            raise RuntimeError(

              'fails with error code {}'.format(process.returncode))

E RuntimeError: fails with error code -6

../wordseg/algos/ag.py:447: RuntimeError
________________________ test_dpseg_parallel_folds[1-1] ________________________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
nfolds = 1, njobs = 1

@pytest.mark.parametrize('nfolds, njobs', [(1, 1), (1, 4), (2, 2), (10, 1)])
def test_dpseg_parallel_folds(prep, nfolds, njobs):
    text = prep[:5]
    if nfolds > 5:
        with pytest.raises(ValueError):
            segment(text, nfolds=nfolds, njobs=njobs)
    else:

      assert len(list(segment(text, nfolds=nfolds, njobs=njobs))) == 5

../test/test_algo_dpseg.py:36:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--ngram 1 --a1 0 --b1 1', log_level = 30
log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
________________________ test_dpseg_parallel_folds[1-4] ________________________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
nfolds = 1, njobs = 4

@pytest.mark.parametrize('nfolds, njobs', [(1, 1), (1, 4), (2, 2), (10, 1)])
def test_dpseg_parallel_folds(prep, nfolds, njobs):
    text = prep[:5]
    if nfolds > 5:
        with pytest.raises(ValueError):
            segment(text, nfolds=nfolds, njobs=njobs)
    else:

      assert len(list(segment(text, nfolds=nfolds, njobs=njobs))) == 5

../test/test_algo_dpseg.py:36:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:789: in call
self.retrieve()

---

self = Parallel(n_jobs=4)

def retrieve(self):
    self._output = list()
    while self._iterating or len(self._jobs) > 0:
        if len(self._jobs) == 0:
            # Wait for an async callback to dispatch new jobs
            time.sleep(0.01)
            continue
        # We need to be careful: the job list can be filling up as
        # we empty it and Python list are not thread-safe by default hence
        # the use of the lock
        with self._lock:
            job = self._jobs.pop(0)

        try:
            if getattr(self._backend, 'supports_timeout', False):
                self._output.extend(job.get(timeout=self.timeout))
            else:
                self._output.extend(job.get())

        except BaseException as exception:
            # Note: we catch any BaseException instead of just Exception
            # instances to also include KeyboardInterrupt.

            # Stop dispatching any new job in the async callback thread
            self._aborting = True

            # If the backend allows it, cancel or kill remaining running
            # tasks without waiting for the results as we will raise
            # the exception we got back to the caller instead of returning
            # any result.
            backend = self._backend
            if (backend is not None and
                    hasattr(backend, 'abort_everything')):
                # If the backend is managed externally we need to make sure
                # to leave it in a working state to allow for future jobs
                # scheduling.
                ensure_ready = self._managed_backend
                backend.abort_everything(ensure_ready=ensure_ready)

            if not isinstance(exception, TransportableException):
                raise
            else:
                # Capture exception to add information on the local
                # stack in addition to the distant stack
                this_report = format_outer_frames(context=10,
                                                  stack_start=1)
                report = """Multiprocessing exception:
%s
---------------------------------------------------------------------------
Sub-process traceback:
---------------------------------------------------------------------------
%s""" % (this_report, exception.message)
                # Convert this to a JoblibException
                exception_type = _mk_exception(exception.etype)[0]
                exception = exception_type(report)

              raise exception

E JoblibRuntimeError: JoblibRuntimeError
E ___________________________________________________________________________
E Multiprocessing exception:
E ...........................................................................
E /anaconda/bin/pytest in ()
E 6 from pkg_resources import load_entry_point
E 7
E 8 if name == 'main':
E 9 sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0])
E 10 sys.exit(
E ---> 11 load_entry_point('pytest==3.0.7', 'console_scripts', 'pytest')()
E 12 )
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/config.py in main(args=None, plugins=None)
E 52 tw.line("ERROR: could not load %s\n" % (e.path), red=True)
E 53 return 4
E 54 else:
E 55 try:
E 56 config.pluginmanager.check_pending()
E ---> 57 return config.hook.pytest_cmdline_main(config=config)
E config.hook.pytest_cmdline_main = <_HookCaller 'pytest_cmdline_main'>
E config = <_pytest.config.Config object>
E 58 finally:
E 59 config._ensure_unconfigure()
E 60 except UsageError as e:
E 61 for msg in e.args:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_cmdline_main'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_cmdline_main'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_cmdline_main'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_cmdline_main'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_cmdline_main'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = None
E hook_impl.function =
E args = [<_pytest.config.Config object>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/main.py in pytest_cmdline_main(config=<_pytest.config.Config object>)
E 122 exitstatus=session.exitstatus)
E 123 config._ensure_unconfigure()
E 124 return session.exitstatus
E 125
E 126 def pytest_cmdline_main(config):
E --> 127 return wrap_session(config, _main)
E config = <_pytest.config.Config object>
E 128
E 129 def _main(config, session):
E 130 """ default command line protocol for initialization, session,
E 131 running tests and reporting. """
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/main.py in wrap_session(config=<_pytest.config.Config object>, doit=)
E 93 try:
E 94 config._do_configure()
E 95 initstate = 1
E 96 config.hook.pytest_sessionstart(session=session)
E 97 initstate = 2
E ---> 98 session.exitstatus = doit(config, session) or 0
E session.exitstatus = 0
E doit =
E config = <_pytest.config.Config object>
E session = <Session 'wordseg_utest'>
E 99 except pytest.UsageError:
E 100 raise
E 101 except KeyboardInterrupt:
E 102 excinfo = _pytest._code.ExceptionInfo()
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/main.py in _main(config=<_pytest.config.Config object>, session=<Session 'wordseg_utest'>)
E 128
E 129 def _main(config, session):
E 130 """ default command line protocol for initialization, session,
E 131 running tests and reporting. """
E 132 config.hook.pytest_collection(session=session)
E --> 133 config.hook.pytest_runtestloop(session=session)
E config.hook.pytest_runtestloop = <_HookCaller 'pytest_runtestloop'>
E session = <Session 'wordseg_utest'>
E 134
E 135 if session.testsfailed:
E 136 return EXIT_TESTSFAILED
E 137 elif session.testscollected == 0:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_runtestloop'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_runtestloop'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_runtestloop'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_runtestloop'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_runtestloop'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = undefined
E hook_impl.function =
E args = [<Session 'wordseg_utest'>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/main.py in pytest_runtestloop(session=<Session 'wordseg_utest'>)
E 149 if session.config.option.collectonly:
E 150 return True
E 151
E 152 for i, item in enumerate(session.items):
E 153 nextitem = session.items[i+1] if i+1 < len(session.items) else None
E --> 154 item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
E item.config.hook.pytest_runtest_protocol = <_HookCaller 'pytest_runtest_protocol'>
E item = <Function 'test_dpseg_parallel_folds[1-4]'>
E nextitem = <Function 'test_dpseg_parallel_folds[2-2]'>
E 155 if session.shouldstop:
E 156 raise session.Interrupted(session.shouldstop)
E 157 return True
E 158
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_runtest_protocol'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[2-2]'>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_runtest_protocol'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[2-2]'>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_runtest_protocol'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[2-2]'>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_runtest_protocol'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[2-2]'>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_runtest_protocol'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[2-2]'>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[2-2]'>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'item': ...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>)
E 608 for argname in hook_impl.argnames:
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E --> 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[1-4]'>]
E self.execute = <bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>
E 614 res = hook_impl.function(*args)
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _wrapped_call(wrap_controller=, func=<bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>)
E 243 """
E 244 try:
E 245 next(wrap_controller) # first yield
E 246 except StopIteration:
E 247 _raise_wrapfail(wrap_controller, "did not yield")
E --> 248 call_outcome = _CallOutcome(func)
E call_outcome = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>
E 249 try:
E 250 wrap_controller.send(call_outcome)
E 251 _raise_wrapfail(wrap_controller, "has second yield")
E 252 except StopIteration:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in init(self=<_pytest.vendored_packages.pluggy._CallOutcome instance>, func=<bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>)
E 260 the exception raised when the function was called. """
E 261 excinfo = None
E 262
E 263 def init(self, func):
E 264 try:
E --> 265 self.result = func()
E self.result = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>
E 266 except BaseException:
E 267 self.excinfo = sys.exc_info()
E 268
E 269 def force_result(self, result):
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'item': ...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = undefined
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[1-4]'>, <Function 'test_dpseg_parallel_folds[2-2]'>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in pytest_runtest_protocol(item=<Function 'test_dpseg_parallel_folds[1-4]'>, nextitem=<Function 'test_dpseg_parallel_folds[2-2]'>)
E 61
E 62 def pytest_runtest_protocol(item, nextitem):
E 63 item.ihook.pytest_runtest_logstart(
E 64 nodeid=item.nodeid, location=item.location,
E 65 )
E ---> 66 runtestprotocol(item, nextitem=nextitem)
E item = <Function 'test_dpseg_parallel_folds[1-4]'>
E nextitem = <Function 'test_dpseg_parallel_folds[2-2]'>
E 67 return True
E 68
E 69 def runtestprotocol(item, log=True, nextitem=None):
E 70 hasrequest = hasattr(item, "_request")
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in runtestprotocol(item=<Function 'test_dpseg_parallel_folds[1-4]'>, log=True, nextitem=<Function 'test_dpseg_parallel_folds[2-2]'>)
E 74 reports = [rep]
E 75 if rep.passed:
E 76 if item.config.option.setupshow:
E 77 show_test_item(item)
E 78 if not item.config.option.setuponly:
E ---> 79 reports.append(call_and_report(item, "call", log))
E reports.append =
E item = <Function 'test_dpseg_parallel_folds[1-4]'>
E log = True
E 80 reports.append(call_and_report(item, "teardown", log,
E 81 nextitem=nextitem))
E 82 # after all teardown hooks have been called
E 83 # want funcargs and request info to go away
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in call_and_report(item=<Function 'test_dpseg_parallel_folds[1-4]'>, when='call', log=True, **kwds={})
E 128
E 129 #
E 130 # Implementation
E 131
E 132 def call_and_report(item, when, log=True, **kwds):
E --> 133 call = call_runtest_hook(item, when, **kwds)
E call = undefined
E item = <Function 'test_dpseg_parallel_folds[1-4]'>
E when = 'call'
E kwds = {}
E 134 hook = item.ihook
E 135 report = hook.pytest_runtest_makereport(item=item, call=call)
E 136 if log:
E 137 hook.pytest_runtest_logreport(report=report)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/pytest/runner.py in call_runtest_hook(item=<Function 'test_dpseg_parallel_folds[1-4]'>, when='call', **kwds={})
E 146 call.excinfo.errisinstance(bdb.BdbQuit))
E 147
E 148 def call_runtest_hook(item, when, **kwds):
E 149 hookname = "pytest_runtest" + when
E 150 ihook = getattr(item.ihook, hookname)
E --> 151 return CallInfo(lambda: ihook(item=item, **kwds), when=when)
E item = <Function 'test_dpseg_parallel_folds[1-4]'>
E kwds = {}
E when = 'call'
E 152
E 153 class CallInfo:
E 154 """ Result/Exception info a function invocation. """
E 155 #: None or ExceptionInfo object.
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in init(self=, func=<function >, when='call')
E 158 #: context of invocation: one of "setup", "call",
E 159 #: "teardown", "memocollect"
E 160 self.when = when
E 161 self.start = time()
E 162 try:
E --> 163 self.result = func()
E self.result = undefined
E func = <function >
E 164 except KeyboardInterrupt:
E 165 self.stop = time()
E 166 raise
E 167 except:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/pytest/runner.py in ()
E 146 call.excinfo.errisinstance(bdb.BdbQuit))
E 147
E 148 def call_runtest_hook(item, when, **kwds):
E 149 hookname = "pytest_runtest" + when
E 150 ihook = getattr(item.ihook, hookname)
E --> 151 return CallInfo(lambda: ihook(item=item, **kwds), when=when)
E 152
E 153 class CallInfo:
E 154 """ Result/Exception info a function invocation. """
E 155 #: None or ExceptionInfo object.
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_runtest_call'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_runtest_call'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_runtest_call'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_runtest_call'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_runtest_call'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[1-4]'>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>)
E 608 for argname in hook_impl.argnames:
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E --> 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E hook_impl.function = <bound method CaptureManager.pytest_runtest_call of <_pytest.capture.CaptureManager instance>>
E args = [<Function 'test_dpseg_parallel_folds[1-4]'>]
E self.execute = <bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>
E 614 res = hook_impl.function(*args)
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _wrapped_call(wrap_controller=, func=<bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>)
E 243 """
E 244 try:
E 245 next(wrap_controller) # first yield
E 246 except StopIteration:
E 247 _raise_wrapfail(wrap_controller, "did not yield")
E --> 248 call_outcome = _CallOutcome(func)
E call_outcome = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>
E 249 try:
E 250 wrap_controller.send(call_outcome)
E 251 _raise_wrapfail(wrap_controller, "has second yield")
E 252 except StopIteration:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in init(self=<_pytest.vendored_packages.pluggy._CallOutcome instance>, func=<bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>)
E 260 the exception raised when the function was called. """
E 261 excinfo = None
E 262
E 263 def init(self, func):
E 264 try:
E --> 265 self.result = func()
E self.result = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>
E 266 except BaseException:
E 267 self.excinfo = sys.exc_info()
E 268
E 269 def force_result(self, result):
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = undefined
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[1-4]'>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in pytest_runtest_call(item=<Function 'test_dpseg_parallel_folds[1-4]'>)
E 99 def pytest_runtest_setup(item):
E 100 item.session._setupstate.prepare(item)
E 101
E 102 def pytest_runtest_call(item):
E 103 try:
E --> 104 item.runtest()
E item.runtest = <bound method Function.runtest of <Function 'test_dpseg_parallel_folds[1-4]'>>
E 105 except Exception:
E 106 # Store trace info to allow postmortem debugging
E 107 type, value, tb = sys.exc_info()
E 108 tb = tb.tb_next # Skip this frame
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/python.py in runtest(self=<Function 'test_dpseg_parallel_folds[1-4]'>)
E 1569 def _isyieldedfunction(self):
E 1570 return getattr(self, "_args", None) is not None
E 1571
E 1572 def runtest(self):
E 1573 """ execute the underlying test function. """
E -> 1574 self.ihook.pytest_pyfunc_call(pyfuncitem=self)
E self.ihook.pytest_pyfunc_call = <_HookCaller 'pytest_pyfunc_call'>
E self = <Function 'test_dpseg_parallel_folds[1-4]'>
E 1575
E 1576 def setup(self):
E 1577 super(Function, self).setup()
E 1578 fixtures.fillfixtures(self)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_pyfunc_call'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[1-4]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[1-4]'>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_pyfunc_call'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[1-4]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[1-4]'>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_pyfunc_call'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[1-4]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[1-4]'>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_pyfunc_call'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[1-4]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[1-4]'>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_pyfunc_call'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[1-4]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[1-4]'>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[1-4]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[1-4]'>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[1-4]'>}>)
E 608 for argname in hook_impl.argnames:
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E --> 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[1-4]'>]
E self.execute = <bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[1-4]'>}>>
E 614 res = hook_impl.function(*args)
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _wrapped_call(wrap_controller=, func=<bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[1-4]'>}>>)
E 243 """
E 244 try:
E 245 next(wrap_controller) # first yield
E 246 except StopIteration:
E 247 _raise_wrapfail(wrap_controller, "did not yield")
E --> 248 call_outcome = _CallOutcome(func)
E call_outcome = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[1-4]'>}>>
E 249 try:
E 250 wrap_controller.send(call_outcome)
E 251 _raise_wrapfail(wrap_controller, "has second yield")
E 252 except StopIteration:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in init(self=<_pytest.vendored_packages.pluggy._CallOutcome instance>, func=<bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[1-4]'>}>>)
E 260 the exception raised when the function was called. """
E 261 excinfo = None
E 262
E 263 def init(self, func):
E 264 try:
E --> 265 self.result = func()
E self.result = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[1-4]'>}>>
E 266 except BaseException:
E 267 self.excinfo = sys.exc_info()
E 268
E 269 def force_result(self, result):
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[1-4]'>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = undefined
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[1-4]'>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/python.py in pytest_pyfunc_call(pyfuncitem=<Function 'test_dpseg_parallel_folds[1-4]'>)
E 149 else:
E 150 funcargs = pyfuncitem.funcargs
E 151 testargs = {}
E 152 for arg in pyfuncitem._fixtureinfo.argnames:
E 153 testargs[arg] = funcargs[arg]
E --> 154 testfunction(**testargs)
E testfunction =
E testargs = {'nfolds': 1, 'njobs': 4, 'prep': ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', 'l ih t ax l b ih t m ao r', 'l ih t ax l b ih t m ao r', 'uw p s iy', 'uw p s iy b ih t iy b ih t iy b ih t iy b ey b iy g er l', 'y uw hh aa s p ae g hh eh t iy ow z', 'w aa n t s ax m m ao r', 'n ow']}
E 155 return True
E 156
E 157 def pytest_collect_file(path, parent):
E 158 ext = path.ext
E
E ...........................................................................
E /Users/acristia/Documents/wordseg_utest/test/test_algo_dpseg.py in test_dpseg_parallel_folds(prep=['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', 'l ih t ax l b ih t m ao r', 'l ih t ax l b ih t m ao r', 'uw p s iy', 'uw p s iy b ih t iy b ih t iy b ih t iy b ey b iy g er l', 'y uw hh aa s p ae g hh eh t iy ow z', 'w aa n t s ax m m ao r', 'n ow'], nfolds=1, njobs=4)
E 31 text = prep[:5]
E 32 if nfolds > 5:
E 33 with pytest.raises(ValueError):
E 34 segment(text, nfolds=nfolds, njobs=njobs)
E 35 else:
E ---> 36 assert len(list(segment(text, nfolds=nfolds, njobs=njobs))) == 5
E text = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s']
E nfolds = 1
E njobs = 4
E 37
E 38
E 39 @pytest.mark.parametrize('args', args)
E 40 def test_dpseg_args(prep, args):
E
E ...........................................................................
E /Users/acristia/Documents/wordseg_utest/wordseg/algos/dpseg.py in segment(text=['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s'], nfolds=1, njobs=4, args='--ngram 1 --a1 0 --b1 1', log=<logging.RootLogger object>)
E 320
E 321 segmented_texts = joblib.Parallel(n_jobs=njobs, verbose=0)(
E 322 joblib.delayed(_dpseg)(
E 323 fold, args, log_level=log.getEffectiveLevel(),
E 324 log_name='wordseg-dpseg - fold {}'.format(n+1))
E --> 325 for n, fold in enumerate(folded_texts))
E folded_texts = [[u'\u0bc4\u0bc3\u0bc5', u'\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', u'\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', u'\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', u'\u0bbe\u0bbc\u0bc1']]
E 326
E 327 log.debug('unfolding the %s folds', nfolds)
E 328 output_text = folding.unfold(segmented_texts, fold_index)
E 329
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py in call(self=Parallel(n_jobs=4), iterable=<generator object >)
E 784 if pre_dispatch == "all" or n_jobs == 1:
E 785 # The iterable was consumed all at once by the above for loop.
E 786 # No need to wait for async callbacks to trigger to
E 787 # consumption.
E 788 self._iterating = False
E --> 789 self.retrieve()
E self.retrieve = <bound method Parallel.retrieve of Parallel(n_jobs=4)>
E 790 # Make sure that we get a last message telling us we are done
E 791 elapsed_time = time.time() - self._start_time
E 792 self._print('Done %3i out of %3i | elapsed: %s finished',
E 793 (len(self._output), len(self._output),
E
E ---------------------------------------------------------------------------
E Sub-process traceback:
E ---------------------------------------------------------------------------
E RuntimeError Mon Jan 15 18:32:45 2018
E PID: 9695 Python 2.7.13: /anaconda/bin/python
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py in call(self=<joblib.parallel.BatchedCalls object>)
E 126 def init(self, iterator_slice):
E 127 self.items = list(iterator_slice)
E 128 self._size = len(self.items)
E 129
E 130 def call(self):
E --> 131 return [func(*args, **kwargs) for func, args, kwargs in self.items]
E func =
E args = ([u'\u0bc4\u0bc3\u0bc5', u'\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', u'\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', u'\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', u'\u0bbe\u0bbc\u0bc1'], '--ngram 1 --a1 0 --b1 1')
E kwargs = {'log_level': 30, 'log_name': 'wordseg-dpseg - fold 1'}
E self.items = [(, ([u'\u0bc4\u0bc3\u0bc5', u'\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', u'\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', u'\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', u'\u0bbe\u0bbc\u0bc1'], '--ngram 1 --a1 0 --b1 1'), {'log_level': 30, 'log_name': 'wordseg-dpseg - fold 1'})]
E 132
E 133 def len(self):
E 134 return self._size
E 135
E
E ...........................................................................
E /Users/acristia/Documents/wordseg_utest/wordseg/algos/dpseg.py in _dpseg(text=[u'\u0bc4\u0bc3\u0bc5', u'\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', u'\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', u'\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', u'\u0bbe\u0bbc\u0bc1'], args='--ngram 1 --a1 0 --b1 1', log_level=30, log_name='wordseg-dpseg - fold 1')
E 288
E 289 thread.join()
E 290 process.wait()
E 291 if process.returncode:
E 292 raise RuntimeError(
E --> 293 'failed with error code {}'.format(process.returncode))
E 294
E 295 tmp_output.seek(0)
E 296 return tmp_output.read().decode('utf8').split('\n')
E 297
E
E RuntimeError: failed with error code -11
E ___________________________________________________________________________

/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:740: JoblibRuntimeError
________________________ test_dpseg_parallel_folds[2-2] ________________________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
nfolds = 2, njobs = 2

@pytest.mark.parametrize('nfolds, njobs', [(1, 1), (1, 4), (2, 2), (10, 1)])
def test_dpseg_parallel_folds(prep, nfolds, njobs):
    text = prep[:5]
    if nfolds > 5:
        with pytest.raises(ValueError):
            segment(text, nfolds=nfolds, njobs=njobs)
    else:

      assert len(list(segment(text, nfolds=nfolds, njobs=njobs))) == 5

../test/test_algo_dpseg.py:36:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:789: in call
self.retrieve()

---

self = Parallel(n_jobs=2)

def retrieve(self):
    self._output = list()
    while self._iterating or len(self._jobs) > 0:
        if len(self._jobs) == 0:
            # Wait for an async callback to dispatch new jobs
            time.sleep(0.01)
            continue
        # We need to be careful: the job list can be filling up as
        # we empty it and Python list are not thread-safe by default hence
        # the use of the lock
        with self._lock:
            job = self._jobs.pop(0)

        try:
            if getattr(self._backend, 'supports_timeout', False):
                self._output.extend(job.get(timeout=self.timeout))
            else:
                self._output.extend(job.get())

        except BaseException as exception:
            # Note: we catch any BaseException instead of just Exception
            # instances to also include KeyboardInterrupt.

            # Stop dispatching any new job in the async callback thread
            self._aborting = True

            # If the backend allows it, cancel or kill remaining running
            # tasks without waiting for the results as we will raise
            # the exception we got back to the caller instead of returning
            # any result.
            backend = self._backend
            if (backend is not None and
                    hasattr(backend, 'abort_everything')):
                # If the backend is managed externally we need to make sure
                # to leave it in a working state to allow for future jobs
                # scheduling.
                ensure_ready = self._managed_backend
                backend.abort_everything(ensure_ready=ensure_ready)

            if not isinstance(exception, TransportableException):
                raise
            else:
                # Capture exception to add information on the local
                # stack in addition to the distant stack
                this_report = format_outer_frames(context=10,
                                                  stack_start=1)
                report = """Multiprocessing exception:
%s
---------------------------------------------------------------------------
Sub-process traceback:
---------------------------------------------------------------------------
%s""" % (this_report, exception.message)
                # Convert this to a JoblibException
                exception_type = _mk_exception(exception.etype)[0]
                exception = exception_type(report)

              raise exception

E JoblibRuntimeError: JoblibRuntimeError
E ___________________________________________________________________________
E Multiprocessing exception:
E ...........................................................................
E /anaconda/bin/pytest in ()
E 6 from pkg_resources import load_entry_point
E 7
E 8 if name == 'main':
E 9 sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0])
E 10 sys.exit(
E ---> 11 load_entry_point('pytest==3.0.7', 'console_scripts', 'pytest')()
E 12 )
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/config.py in main(args=None, plugins=None)
E 52 tw.line("ERROR: could not load %s\n" % (e.path), red=True)
E 53 return 4
E 54 else:
E 55 try:
E 56 config.pluginmanager.check_pending()
E ---> 57 return config.hook.pytest_cmdline_main(config=config)
E config.hook.pytest_cmdline_main = <_HookCaller 'pytest_cmdline_main'>
E config = <_pytest.config.Config object>
E 58 finally:
E 59 config._ensure_unconfigure()
E 60 except UsageError as e:
E 61 for msg in e.args:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_cmdline_main'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_cmdline_main'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>, <_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_cmdline_main'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_cmdline_main'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_cmdline_main'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>, 'config': <_pytest.config.Config object>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'__multi...: <_pytest.config.Config object at 0x107367f10>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = None
E hook_impl.function =
E args = [<_pytest.config.Config object>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/main.py in pytest_cmdline_main(config=<_pytest.config.Config object>)
E 122 exitstatus=session.exitstatus)
E 123 config._ensure_unconfigure()
E 124 return session.exitstatus
E 125
E 126 def pytest_cmdline_main(config):
E --> 127 return wrap_session(config, _main)
E config = <_pytest.config.Config object>
E 128
E 129 def _main(config, session):
E 130 """ default command line protocol for initialization, session,
E 131 running tests and reporting. """
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/main.py in wrap_session(config=<_pytest.config.Config object>, doit=)
E 93 try:
E 94 config._do_configure()
E 95 initstate = 1
E 96 config.hook.pytest_sessionstart(session=session)
E 97 initstate = 2
E ---> 98 session.exitstatus = doit(config, session) or 0
E session.exitstatus = 0
E doit =
E config = <_pytest.config.Config object>
E session = <Session 'wordseg_utest'>
E 99 except pytest.UsageError:
E 100 raise
E 101 except KeyboardInterrupt:
E 102 excinfo = _pytest._code.ExceptionInfo()
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/main.py in _main(config=<_pytest.config.Config object>, session=<Session 'wordseg_utest'>)
E 128
E 129 def _main(config, session):
E 130 """ default command line protocol for initialization, session,
E 131 running tests and reporting. """
E 132 config.hook.pytest_collection(session=session)
E --> 133 config.hook.pytest_runtestloop(session=session)
E config.hook.pytest_runtestloop = <_HookCaller 'pytest_runtestloop'>
E session = <Session 'wordseg_utest'>
E 134
E 135 if session.testsfailed:
E 136 return EXIT_TESTSFAILED
E 137 elif session.testscollected == 0:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_runtestloop'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_runtestloop'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_runtestloop'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_runtestloop'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_runtestloop'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>, 'session': <Session 'wordseg_utest'>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'__multi...gs={...}>, 'session': <Session 'wordseg_utest'>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = undefined
E hook_impl.function =
E args = [<Session 'wordseg_utest'>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/main.py in pytest_runtestloop(session=<Session 'wordseg_utest'>)
E 149 if session.config.option.collectonly:
E 150 return True
E 151
E 152 for i, item in enumerate(session.items):
E 153 nextitem = session.items[i+1] if i+1 < len(session.items) else None
E --> 154 item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
E item.config.hook.pytest_runtest_protocol = <_HookCaller 'pytest_runtest_protocol'>
E item = <Function 'test_dpseg_parallel_folds[2-2]'>
E nextitem = <Function 'test_dpseg_parallel_folds[10-1]'>
E 155 if session.shouldstop:
E 156 raise session.Interrupted(session.shouldstop)
E 157 return True
E 158
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_runtest_protocol'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...': <Function 'test_dpseg_parallel_folds[10-1]'>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[10-1]'>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_runtest_protocol'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...': <Function 'test_dpseg_parallel_folds[10-1]'>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[10-1]'>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_runtest_protocol'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...': <Function 'test_dpseg_parallel_folds[10-1]'>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[10-1]'>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_runtest_protocol'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...': <Function 'test_dpseg_parallel_folds[10-1]'>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[10-1]'>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_runtest_protocol'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...': <Function 'test_dpseg_parallel_folds[10-1]'>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[10-1]'>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...': <Function 'test_dpseg_parallel_folds[10-1]'>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>, 'nextitem': <Function 'test_dpseg_parallel_folds[10-1]'>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'item': ...': <Function 'test_dpseg_parallel_folds[10-1]'>}>)
E 608 for argname in hook_impl.argnames:
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E --> 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[2-2]'>]
E self.execute = <bound method _MultiCall.execute of <_MultiCall ...: <Function 'test_dpseg_parallel_folds[10-1]'>}>>
E 614 res = hook_impl.function(*args)
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _wrapped_call(wrap_controller=, func=<bound method _MultiCall.execute of <_MultiCall ...: <Function 'test_dpseg_parallel_folds[10-1]'>}>>)
E 243 """
E 244 try:
E 245 next(wrap_controller) # first yield
E 246 except StopIteration:
E 247 _raise_wrapfail(wrap_controller, "did not yield")
E --> 248 call_outcome = _CallOutcome(func)
E call_outcome = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ...: <Function 'test_dpseg_parallel_folds[10-1]'>}>>
E 249 try:
E 250 wrap_controller.send(call_outcome)
E 251 _raise_wrapfail(wrap_controller, "has second yield")
E 252 except StopIteration:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in init(self=<_pytest.vendored_packages.pluggy._CallOutcome instance>, func=<bound method _MultiCall.execute of <_MultiCall ...: <Function 'test_dpseg_parallel_folds[10-1]'>}>>)
E 260 the exception raised when the function was called. """
E 261 excinfo = None
E 262
E 263 def init(self, func):
E 264 try:
E --> 265 self.result = func()
E self.result = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ...: <Function 'test_dpseg_parallel_folds[10-1]'>}>>
E 266 except BaseException:
E 267 self.excinfo = sys.exc_info()
E 268
E 269 def force_result(self, result):
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'item': ...': <Function 'test_dpseg_parallel_folds[10-1]'>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = undefined
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[2-2]'>, <Function 'test_dpseg_parallel_folds[10-1]'>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in pytest_runtest_protocol(item=<Function 'test_dpseg_parallel_folds[2-2]'>, nextitem=<Function 'test_dpseg_parallel_folds[10-1]'>)
E 61
E 62 def pytest_runtest_protocol(item, nextitem):
E 63 item.ihook.pytest_runtest_logstart(
E 64 nodeid=item.nodeid, location=item.location,
E 65 )
E ---> 66 runtestprotocol(item, nextitem=nextitem)
E item = <Function 'test_dpseg_parallel_folds[2-2]'>
E nextitem = <Function 'test_dpseg_parallel_folds[10-1]'>
E 67 return True
E 68
E 69 def runtestprotocol(item, log=True, nextitem=None):
E 70 hasrequest = hasattr(item, "_request")
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in runtestprotocol(item=<Function 'test_dpseg_parallel_folds[2-2]'>, log=True, nextitem=<Function 'test_dpseg_parallel_folds[10-1]'>)
E 74 reports = [rep]
E 75 if rep.passed:
E 76 if item.config.option.setupshow:
E 77 show_test_item(item)
E 78 if not item.config.option.setuponly:
E ---> 79 reports.append(call_and_report(item, "call", log))
E reports.append =
E item = <Function 'test_dpseg_parallel_folds[2-2]'>
E log = True
E 80 reports.append(call_and_report(item, "teardown", log,
E 81 nextitem=nextitem))
E 82 # after all teardown hooks have been called
E 83 # want funcargs and request info to go away
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in call_and_report(item=<Function 'test_dpseg_parallel_folds[2-2]'>, when='call', log=True, **kwds={})
E 128
E 129 #
E 130 # Implementation
E 131
E 132 def call_and_report(item, when, log=True, **kwds):
E --> 133 call = call_runtest_hook(item, when, **kwds)
E call = undefined
E item = <Function 'test_dpseg_parallel_folds[2-2]'>
E when = 'call'
E kwds = {}
E 134 hook = item.ihook
E 135 report = hook.pytest_runtest_makereport(item=item, call=call)
E 136 if log:
E 137 hook.pytest_runtest_logreport(report=report)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/pytest/runner.py in call_runtest_hook(item=<Function 'test_dpseg_parallel_folds[2-2]'>, when='call', **kwds={})
E 146 call.excinfo.errisinstance(bdb.BdbQuit))
E 147
E 148 def call_runtest_hook(item, when, **kwds):
E 149 hookname = "pytest_runtest" + when
E 150 ihook = getattr(item.ihook, hookname)
E --> 151 return CallInfo(lambda: ihook(item=item, **kwds), when=when)
E item = <Function 'test_dpseg_parallel_folds[2-2]'>
E kwds = {}
E when = 'call'
E 152
E 153 class CallInfo:
E 154 """ Result/Exception info a function invocation. """
E 155 #: None or ExceptionInfo object.
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in init(self=, func=<function >, when='call')
E 158 #: context of invocation: one of "setup", "call",
E 159 #: "teardown", "memocollect"
E 160 self.when = when
E 161 self.start = time()
E 162 try:
E --> 163 self.result = func()
E self.result = undefined
E func = <function >
E 164 except KeyboardInterrupt:
E 165 self.stop = time()
E 166 raise
E 167 except:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/pytest/runner.py in ()
E 146 call.excinfo.errisinstance(bdb.BdbQuit))
E 147
E 148 def call_runtest_hook(item, when, **kwds):
E 149 hookname = "pytest_runtest" + when
E 150 ihook = getattr(item.ihook, hookname)
E --> 151 return CallInfo(lambda: ihook(item=item, **kwds), when=when)
E 152
E 153 class CallInfo:
E 154 """ Result/Exception info a function invocation. """
E 155 #: None or ExceptionInfo object.
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_runtest_call'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_runtest_call'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_runtest_call'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_runtest_call'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_runtest_call'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>, 'item': <Function 'test_dpseg_parallel_folds[2-2]'>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>)
E 608 for argname in hook_impl.argnames:
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E --> 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E hook_impl.function = <bound method CaptureManager.pytest_runtest_call of <_pytest.capture.CaptureManager instance>>
E args = [<Function 'test_dpseg_parallel_folds[2-2]'>]
E self.execute = <bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>
E 614 res = hook_impl.function(*args)
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _wrapped_call(wrap_controller=, func=<bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>)
E 243 """
E 244 try:
E 245 next(wrap_controller) # first yield
E 246 except StopIteration:
E 247 _raise_wrapfail(wrap_controller, "did not yield")
E --> 248 call_outcome = _CallOutcome(func)
E call_outcome = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>
E 249 try:
E 250 wrap_controller.send(call_outcome)
E 251 _raise_wrapfail(wrap_controller, "has second yield")
E 252 except StopIteration:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in init(self=<_pytest.vendored_packages.pluggy._CallOutcome instance>, func=<bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>)
E 260 the exception raised when the function was called. """
E 261 excinfo = None
E 262
E 263 def init(self, func):
E 264 try:
E --> 265 self.result = func()
E self.result = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ... <_MultiCall 0 results, 0 meths, kwargs={...}>}>>
E 266 except BaseException:
E 267 self.excinfo = sys.exc_info()
E 268
E 269 def force_result(self, result):
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'item': ...: <_MultiCall 0 results, 0 meths, kwargs={...}>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = undefined
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[2-2]'>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/runner.py in pytest_runtest_call(item=<Function 'test_dpseg_parallel_folds[2-2]'>)
E 99 def pytest_runtest_setup(item):
E 100 item.session._setupstate.prepare(item)
E 101
E 102 def pytest_runtest_call(item):
E 103 try:
E --> 104 item.runtest()
E item.runtest = <bound method Function.runtest of <Function 'test_dpseg_parallel_folds[2-2]'>>
E 105 except Exception:
E 106 # Store trace info to allow postmortem debugging
E 107 type, value, tb = sys.exc_info()
E 108 tb = tb.tb_next # Skip this frame
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/python.py in runtest(self=<Function 'test_dpseg_parallel_folds[2-2]'>)
E 1569 def _isyieldedfunction(self):
E 1570 return getattr(self, "_args", None) is not None
E 1571
E 1572 def runtest(self):
E 1573 """ execute the underlying test function. """
E -> 1574 self.ihook.pytest_pyfunc_call(pyfuncitem=self)
E self.ihook.pytest_pyfunc_call = <_HookCaller 'pytest_pyfunc_call'>
E self = <Function 'test_dpseg_parallel_folds[2-2]'>
E 1575
E 1576 def setup(self):
E 1577 super(Function, self).setup()
E 1578 fixtures.fillfixtures(self)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in call(self=<_HookCaller 'pytest_pyfunc_call'>, **kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[2-2]'>})
E 740 def repr(self):
E 741 return "<_HookCaller %r>" % (self.name,)
E 742
E 743 def call(self, **kwargs):
E 744 assert not self.is_historic()
E --> 745 return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
E self._hookexec = <bound method PytestPluginManager._hookexec of <_pytest.config.PytestPluginManager object>>
E self = <_HookCaller 'pytest_pyfunc_call'>
E self._nonwrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E self._wrappers = [<_pytest.vendored_packages.pluggy.HookImpl instance>]
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[2-2]'>}
E 746
E 747 def call_historic(self, proc=None, kwargs=None):
E 748 self._call_history.append((kwargs or {}, proc))
E 749 # historizing hooks don't return results
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _hookexec(self=<_pytest.config.PytestPluginManager object>, hook=<_HookCaller 'pytest_pyfunc_call'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[2-2]'>})
E 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E --> 339 return self._inner_hookexec(hook, methods, kwargs)
E self._inner_hookexec = <function >
E hook = <_HookCaller 'pytest_pyfunc_call'>
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[2-2]'>}
E 340
E 341 def register(self, plugin, name=None):
E 342 """ Register a plugin and return its canonical name or None if the name
E 343 is blocked from registering. Raise a ValueError if the plugin is already
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in (hook=<_HookCaller 'pytest_pyfunc_call'>, methods=[], kwargs={'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[2-2]'>})
E 329 self._plugin_distinfo = []
E 330 self.trace = _TagTracer().get("pluginmanage")
E 331 self.hook = _HookRelay(self.trace.root.get("hook"))
E 332 self._implprefix = implprefix
E 333 self._inner_hookexec = lambda hook, methods, kwargs:
E --> 334 _MultiCall(methods, kwargs, hook.spec_opts).execute()
E methods = []
E kwargs = {'multicall': <_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>, 'pyfuncitem': <Function 'test_dpseg_parallel_folds[2-2]'>}
E hook.spec_opts.execute = undefined
E 335
E 336 def _hookexec(self, hook, methods, kwargs):
E 337 # called from all hookcaller instances.
E 338 # enable_tracing will set its own wrapping function at self._inner_hookexec
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>)
E 608 for argname in hook_impl.argnames:
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E --> 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[2-2]'>]
E self.execute = <bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>
E 614 res = hook_impl.function(*args)
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in _wrapped_call(wrap_controller=, func=<bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>)
E 243 """
E 244 try:
E 245 next(wrap_controller) # first yield
E 246 except StopIteration:
E 247 _raise_wrapfail(wrap_controller, "did not yield")
E --> 248 call_outcome = _CallOutcome(func)
E call_outcome = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>
E 249 try:
E 250 wrap_controller.send(call_outcome)
E 251 _raise_wrapfail(wrap_controller, "has second yield")
E 252 except StopIteration:
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in init(self=<_pytest.vendored_packages.pluggy._CallOutcome instance>, func=<bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>)
E 260 the exception raised when the function was called. """
E 261 excinfo = None
E 262
E 263 def init(self, func):
E 264 try:
E --> 265 self.result = func()
E self.result = undefined
E func = <bound method _MultiCall.execute of <_MultiCall ...': <Function 'test_dpseg_parallel_folds[2-2]'>}>>
E 266 except BaseException:
E 267 self.excinfo = sys.exc_info()
E 268
E 269 def force_result(self, result):
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py in execute(self=<_MultiCall 0 results, 0 meths, kwargs={'__multi...m': <Function 'test_dpseg_parallel_folds[2-2]'>}>)
E 609 if argname not in all_kwargs:
E 610 raise HookCallError(
E 611 "hook call must provide argument %r" % (argname,))
E 612 if hook_impl.hookwrapper:
E 613 return _wrapped_call(hook_impl.function(*args), self.execute)
E --> 614 res = hook_impl.function(*args)
E res = undefined
E hook_impl.function =
E args = [<Function 'test_dpseg_parallel_folds[2-2]'>]
E 615 if res is not None:
E 616 if firstresult:
E 617 return res
E 618 results.append(res)
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/_pytest/python.py in pytest_pyfunc_call(pyfuncitem=<Function 'test_dpseg_parallel_folds[2-2]'>)
E 149 else:
E 150 funcargs = pyfuncitem.funcargs
E 151 testargs = {}
E 152 for arg in pyfuncitem._fixtureinfo.argnames:
E 153 testargs[arg] = funcargs[arg]
E --> 154 testfunction(**testargs)
E testfunction =
E testargs = {'nfolds': 2, 'njobs': 2, 'prep': ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', 'l ih t ax l b ih t m ao r', 'l ih t ax l b ih t m ao r', 'uw p s iy', 'uw p s iy b ih t iy b ih t iy b ih t iy b ey b iy g er l', 'y uw hh aa s p ae g hh eh t iy ow z', 'w aa n t s ax m m ao r', 'n ow']}
E 155 return True
E 156
E 157 def pytest_collect_file(path, parent):
E 158 ext = path.ext
E
E ...........................................................................
E /Users/acristia/Documents/wordseg_utest/test/test_algo_dpseg.py in test_dpseg_parallel_folds(prep=['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', 'l ih t ax l b ih t m ao r', 'l ih t ax l b ih t m ao r', 'uw p s iy', 'uw p s iy b ih t iy b ih t iy b ih t iy b ey b iy g er l', 'y uw hh aa s p ae g hh eh t iy ow z', 'w aa n t s ax m m ao r', 'n ow'], nfolds=2, njobs=2)
E 31 text = prep[:5]
E 32 if nfolds > 5:
E 33 with pytest.raises(ValueError):
E 34 segment(text, nfolds=nfolds, njobs=njobs)
E 35 else:
E ---> 36 assert len(list(segment(text, nfolds=nfolds, njobs=njobs))) == 5
E text = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s']
E nfolds = 2
E njobs = 2
E 37
E 38
E 39 @pytest.mark.parametrize('args', args)
E 40 def test_dpseg_args(prep, args):
E
E ...........................................................................
E /Users/acristia/Documents/wordseg_utest/wordseg/algos/dpseg.py in segment(text=['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s'], nfolds=2, njobs=2, args='--ngram 1 --a1 0 --b1 1', log=<logging.RootLogger object>)
E 320
E 321 segmented_texts = joblib.Parallel(n_jobs=njobs, verbose=0)(
E 322 joblib.delayed(_dpseg)(
E 323 fold, args, log_level=log.getEffectiveLevel(),
E 324 log_name='wordseg-dpseg - fold {}'.format(n+1))
E --> 325 for n, fold in enumerate(folded_texts))
E folded_texts = [[u'\u0bc4\u0bc3\u0bc5', u'\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', u'\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', u'\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', u'\u0bbe\u0bbc\u0bc1'], [u'\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', u'\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', u'\u0bbe\u0bbc\u0bc1', u'\u0bc4\u0bc3\u0bc5', u'\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9']]
E 326
E 327 log.debug('unfolding the %s folds', nfolds)
E 328 output_text = folding.unfold(segmented_texts, fold_index)
E 329
E
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py in call(self=Parallel(n_jobs=2), iterable=<generator object >)
E 784 if pre_dispatch == "all" or n_jobs == 1:
E 785 # The iterable was consumed all at once by the above for loop.
E 786 # No need to wait for async callbacks to trigger to
E 787 # consumption.
E 788 self._iterating = False
E --> 789 self.retrieve()
E self.retrieve = <bound method Parallel.retrieve of Parallel(n_jobs=2)>
E 790 # Make sure that we get a last message telling us we are done
E 791 elapsed_time = time.time() - self._start_time
E 792 self._print('Done %3i out of %3i | elapsed: %s finished',
E 793 (len(self._output), len(self._output),
E
E ---------------------------------------------------------------------------
E Sub-process traceback:
E ---------------------------------------------------------------------------
E RuntimeError Mon Jan 15 18:32:45 2018
E PID: 9700 Python 2.7.13: /anaconda/bin/python
E ...........................................................................
E /anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py in call(self=<joblib.parallel.BatchedCalls object>)
E 126 def init(self, iterator_slice):
E 127 self.items = list(iterator_slice)
E 128 self._size = len(self.items)
E 129
E 130 def call(self):
E --> 131 return [func(*args, **kwargs) for func, args, kwargs in self.items]
E func =
E args = ([u'\u0bc4\u0bc3\u0bc5', u'\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', u'\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', u'\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', u'\u0bbe\u0bbc\u0bc1'], '--ngram 1 --a1 0 --b1 1')
E kwargs = {'log_level': 30, 'log_name': 'wordseg-dpseg - fold 1'}
E self.items = [(, ([u'\u0bc4\u0bc3\u0bc5', u'\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', u'\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', u'\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', u'\u0bbe\u0bbc\u0bc1'], '--ngram 1 --a1 0 --b1 1'), {'log_level': 30, 'log_name': 'wordseg-dpseg - fold 1'})]
E 132
E 133 def len(self):
E 134 return self._size
E 135
E
E ...........................................................................
E /Users/acristia/Documents/wordseg_utest/wordseg/algos/dpseg.py in _dpseg(text=[u'\u0bc4\u0bc3\u0bc5', u'\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', u'\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', u'\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', u'\u0bbe\u0bbc\u0bc1'], args='--ngram 1 --a1 0 --b1 1', log_level=30, log_name='wordseg-dpseg - fold 1')
E 288
E 289 thread.join()
E 290 process.wait()
E 291 if process.returncode:
E 292 raise RuntimeError(
E --> 293 'failed with error code {}'.format(process.returncode))
E 294
E 295 tmp_output.seek(0)
E 296 return tmp_output.read().decode('utf8').split('\n')
E 297
E
E RuntimeError: failed with error code -11
E ___________________________________________________________________________

/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:740: JoblibRuntimeError
______________________________ test_dpseg_args[] _______________________________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
args = ''

@pytest.mark.parametrize('args', args)
def test_dpseg_args(prep, args):

  segmented = segment(prep[:5], nfolds=1, args=args)

../test/test_algo_dpseg.py:41:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1'], args = ''
log_level = 30, log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
___________ test_dpseg_args[--ngram 1 --a1 0 --b1 1 --a2 0 -- b2 1] ___________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
args = '--ngram 1 --a1 0 --b1 1 --a2 0 -- b2 1'

@pytest.mark.parametrize('args', args)
def test_dpseg_args(prep, args):

  segmented = segment(prep[:5], nfolds=1, args=args)

../test/test_algo_dpseg.py:41:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--ngram 1 --a1 0 --b1 1 --a2 0 -- b2 1', log_level = 30
log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
___________ test_dpseg_args[--ngram 1 --a1 0.1 --b1 0.9 --do-mbdp 1] ___________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
args = '--ngram 1 --a1 0.1 --b1 0.9 --do-mbdp 1'

@pytest.mark.parametrize('args', args)
def test_dpseg_args(prep, args):

  segmented = segment(prep[:5], nfolds=1, args=args)

../test/test_algo_dpseg.py:41:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--ngram 1 --a1 0.1 --b1 0.9 --do-mbdp 1', log_level = 30
log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
__________ test_dpseg_args[--ngram 1 --a1 0 --b1 1 --forget-method P] __________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
args = '--ngram 1 --a1 0 --b1 1 --forget-method P'

@pytest.mark.parametrize('args', args)
def test_dpseg_args(prep, args):

  segmented = segment(prep[:5], nfolds=1, args=args)

../test/test_algo_dpseg.py:41:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--ngram 1 --a1 0 --b1 1 --forget-method P', log_level = 30
log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
_____ test_dpseg_args[--ngram 1 --a1 0 --b1 1 --estimator V --mode batch] ______

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
args = '--ngram 1 --a1 0 --b1 1 --estimator V --mode batch'

@pytest.mark.parametrize('args', args)
def test_dpseg_args(prep, args):

  segmented = segment(prep[:5], nfolds=1, args=args)

../test/test_algo_dpseg.py:41:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--ngram 1 --a1 0 --b1 1 --estimator V --mode batch', log_level = 30
log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
____________ test_dpseg_args[--ngram 1 --a1 0 --b1 1 --estimator F] ____________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
args = '--ngram 1 --a1 0 --b1 1 --estimator F'

@pytest.mark.parametrize('args', args)
def test_dpseg_args(prep, args):

  segmented = segment(prep[:5], nfolds=1, args=args)

../test/test_algo_dpseg.py:41:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--ngram 1 --a1 0 --b1 1 --estimator F', log_level = 30
log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
____________ test_dpseg_args[--ngram 1 --a1 0 --b1 1 --estimator T] ____________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
args = '--ngram 1 --a1 0 --b1 1 --estimator T'

@pytest.mark.parametrize('args', args)
def test_dpseg_args(prep, args):

  segmented = segment(prep[:5], nfolds=1, args=args)

../test/test_algo_dpseg.py:41:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--ngram 1 --a1 0 --b1 1 --estimator T', log_level = 30
log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
test_dpseg_args[--ngram 1 --a1 0 --b1 1 --estimator D --mode online --eval-maximize 1 --eval-interval 50 --decay-rate 1.5 --samples-per-utt 20 --hypersamp-ratio 0 --anneal-a 10 --burnin-iterations 1 --anneal-iterations 0]

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
args = '--ngram 1 --a1 0 --b1 1 --estimator D --mode online --eval-maximize 1 --eval-interval 50 --decay-rate 1.5 --samples-per-utt 20 --hypersamp-ratio 0 --anneal-a 10 --burnin-iterations 1 --anneal-iterations 0'

@pytest.mark.parametrize('args', args)
def test_dpseg_args(prep, args):

  segmented = segment(prep[:5], nfolds=1, args=args)

../test/test_algo_dpseg.py:41:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--ngram 1 --a1 0 --b1 1 --estimator D --mode online --eval-maximize 1 --eval-interval 50 --decay-rate 1.5 --samples-per-utt 20 --hypersamp-ratio 0 --anneal-a 10 --burnin-iterations 1 --anneal-iterations 0'
log_level = 30, log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
test_dpseg_from_config_file[/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/bi_dmcmc.conf]

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
conf = '/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/bi_dmcmc.conf'

@pytest.mark.parametrize(
    'conf', (f for f in wordseg.utils.get_config_files('dpseg')
             if 'bi_ideal' not in f))
def test_dpseg_from_config_file(prep, conf):
    segmented = segment(

      prep[:5], nfolds=1, args='--config-file {}'.format(conf))

../test/test_algo_dpseg.py:60:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--config-file /Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/bi_dmcmc.conf'
log_level = 30, log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
test_dpseg_from_config_file[/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/dps.conf]

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
conf = '/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/dps.conf'

@pytest.mark.parametrize(
    'conf', (f for f in wordseg.utils.get_config_files('dpseg')
             if 'bi_ideal' not in f))
def test_dpseg_from_config_file(prep, conf):
    segmented = segment(

      prep[:5], nfolds=1, args='--config-file {}'.format(conf))

../test/test_algo_dpseg.py:60:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--config-file /Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/dps.conf'
log_level = 30, log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
test_dpseg_from_config_file[/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/viterbi.conf]

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
conf = '/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/viterbi.conf'

@pytest.mark.parametrize(
    'conf', (f for f in wordseg.utils.get_config_files('dpseg')
             if 'bi_ideal' not in f))
def test_dpseg_from_config_file(prep, conf):
    segmented = segment(

      prep[:5], nfolds=1, args='--config-file {}'.format(conf))

../test/test_algo_dpseg.py:60:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc4\u0bc3\u0bc5', '\u0bbb\u0bba\u0bc5\u0bc8\u0bb9\u0bc6\u0bc9', '\u0bc1\u0bc9\u0bca\u0bc0\u0bbc\u0bbf\u0bc2\u0bca\u0bcb\u0bc7', '\u0bc8\u0bbd\u0bc7\u0bcd\u0bc3\u0bcd\u0bcc\u0bbe\u0bb9\u0bc7\u0bbd\u0bbc', '\u0bbe\u0bbc\u0bc1']
args = '--config-file /Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/dpseg/viterbi.conf'
log_level = 30, log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
___________________________ test_pipeline[ag-ascii] ____________________________

algo = 'ag', encoding = 'ascii'
tags = ['hh ih r ;esyll ;eword', 'dh eh r ;esyll ;eword w iy ;esyll ;eword g ow ;esyll ;eword', 's ow ;esyll ;eword ax m p ;e...yll ;eword', 'uw p s ;esyll ;eword', 'l ih t ;esyll ax l ;esyll ;eword b ih t ;esyll ;eword m ao r ;esyll ;eword', ...]
tmpdir = local('/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_pipeline_ag_ascii_0')

@pytest.mark.parametrize('algo, encoding', params)
def test_pipeline(algo, encoding, tags, tmpdir):
    # the token separator we use in the whole pipeline
    separator = Separator(phone=' ', syllable=';esyll', word=';eword')

    # add some unicode chars in the input text
    if encoding == 'unicode':
        tags = add_unicode(tags)

    # build the gold version from the tags
    gold = list(wordseg.prepare.gold(tags, separator=separator))
    assert len(gold) == len(tags)
    for a, b in zip(gold, tags):
        assert separator.remove(a) == separator.remove(b)

    # prepare the text for segmentation
    prepared_text = list(wordseg.prepare.prepare(tags, separator=separator))
    assert len(prepared_text) == len(tags)
    for a, b in zip(prepared_text, tags):
        assert separator.remove(a) == separator.remove(b)

    # segment it with the given algo (use default options)
    if algo in ('dpseg', 'puddle'):
        # only 1 fold for iterative algos: faster
        segmented = list(algos[algo].segment(prepared_text, nfolds=1))
    elif algo == 'ag':
        # add grammar related arguments, if in unicode test adapt the
        # grammar too
        grammar_file = os.path.join(
            os.path.dirname(wordseg.algos.ag.get_grammar_files()[0]),
            'Colloc0_enFestival.lt')
        if encoding == 'unicode':
            grammar_unicode = add_unicode(
                codecs.open(grammar_file, 'r', encoding='utf8'))
            grammar_file = os.path.join(str(tmpdir), 'grammar.lt')
            codecs.open(grammar_file, 'w', encoding='utf8').write(
                '\n'.join(grammar_unicode))
        segmented = list(algos[algo].segment(

          prepared_text, grammar_file, 'Colloc0', nruns=1))

../test/test_pipeline.py:77:

---

../wordseg/algos/ag.py:719: in segment
for n in range(nruns))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
grammar_file = '/Users/acristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py2.7.egg-tmp/data/ag/Colloc0_enFestival.lt'
args = '-E -d 0 -a 0.0001 -b 10000 -e 1 -f 1 -g 100 -h 0.01 -R -1 -P -x 10 -r 11361'
test_text = 'hh ih r\ndh eh r w iy g ow\ns ow ax m p ey sh ax n t\nw ah t d ih d y uw iy t ah p\nuw p s\nl ih t ax l b ih t m ao r...iy b ih t iy b ih t iy b ih t iy b ey b iy g er l\ny uw hh aa s p ae g hh eh t iy ow z\nw aa n t s ax m m ao r\nn ow\n'
log_level = 30, log_name = 'wordseg-ag - run 1'

def _run_ag_single(text, grammar_file, args, test_text=None,
                   log_level=logging.ERROR, log_name='wordseg-ag'):
    """Runs the AG program a single time and returns the computed parse trees

    Parameters
    ----------
    text : sequence
        The list of utterances to train the model on, and to segment
        if `test_text` is None.
    grammar_file : str
        The path to the grammar file to use for segmentation
    args : str
        Command line options to run the AG program with, use
        'wordseg-ag --help' to have a complete list of available
        options
    test_text : sequence, optional
        If not None, the list of utterances to segment on the model
        learned from `text`
    log : logging.Logger, optional
        A logger where to send log messages

    Returns
    -------
    A list of parse trees, one tree per input utterance in `text`

    Raises
    ------
    RuntimeError
        If the AG program fails and returns an error code

    """
    log = utils.get_logger(name=log_name, level=log_level)

    # setup the train text as a multiline string
    train_text = '\n'.join(utt.strip() for utt in text) + '\n'

    # setup the test text as well
    test_text = train_text if test_text is None else (
        '\n'.join(utt.strip() for utt in test_text) + '\n')

    # we need to write the test text as a tempfile, so we create a
    # tempdir. The directory and its content is automatically erased
    # when done
    temp_dir = tempfile.mkdtemp()
    try:
        # write the test text as a temporary file. ylt extension is
        # the one used in the original AG implementation
        test_tmpfile = os.path.join(temp_dir, 'test.ylt')
        codecs.open(test_tmpfile, 'w', encoding='utf8').write(test_text)

        # generate the command to run as a subprocess
        command = ('{binary} {grammar} {args} -u {test} '.format(
            binary=utils.get_binary('ag'),
            grammar=grammar_file,
            args=args,
            test=test_tmpfile))

        log.info('running %s', command)

        # run the command as a subprocess
        process = subprocess.Popen(
            shlex.split(command),
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        parses, messages = process.communicate(train_text.encode('utf8'))

        # log.debug the AG messages AFTER EXECUTION
        messages = messages.decode('utf8').split('\n')
        for msg in messages:
            log.debug(re.sub('^# ', '', msg).strip())

        # fail if AG returns an error code
        if process.returncode:
            raise RuntimeError(

              'fails with error code {}'.format(process.returncode))

E RuntimeError: fails with error code -6

../wordseg/algos/ag.py:447: RuntimeError
__________________________ test_pipeline[ag-unicode] ___________________________

algo = 'ag', encoding = 'unicode'
tags = ['\u10da \u0b16 r ;esyll ;eword', 'dh \u0807 r ;esyll ;eword w iy ;esyll ;eword g ow ;esyll ;eword', 's ow ;esyll ;eword ax m p ;esyl...esyll ;eword', 'uw p s ;esyll ;eword', 'l \u0b16 t ;esyll ax l ;esyll ;eword b \u0b16 t ;esyll ;eword m ao r ;esyll ;eword', ...]
tmpdir = local('/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_pipeline_ag_unicode_0')

@pytest.mark.parametrize('algo, encoding', params)
def test_pipeline(algo, encoding, tags, tmpdir):
    # the token separator we use in the whole pipeline
    separator = Separator(phone=' ', syllable=';esyll', word=';eword')

    # add some unicode chars in the input text
    if encoding == 'unicode':
        tags = add_unicode(tags)

    # build the gold version from the tags
    gold = list(wordseg.prepare.gold(tags, separator=separator))
    assert len(gold) == len(tags)
    for a, b in zip(gold, tags):
        assert separator.remove(a) == separator.remove(b)

    # prepare the text for segmentation
    prepared_text = list(wordseg.prepare.prepare(tags, separator=separator))
    assert len(prepared_text) == len(tags)
    for a, b in zip(prepared_text, tags):
        assert separator.remove(a) == separator.remove(b)

    # segment it with the given algo (use default options)
    if algo in ('dpseg', 'puddle'):
        # only 1 fold for iterative algos: faster
        segmented = list(algos[algo].segment(prepared_text, nfolds=1))
    elif algo == 'ag':
        # add grammar related arguments, if in unicode test adapt the
        # grammar too
        grammar_file = os.path.join(
            os.path.dirname(wordseg.algos.ag.get_grammar_files()[0]),
            'Colloc0_enFestival.lt')
        if encoding == 'unicode':
            grammar_unicode = add_unicode(
                codecs.open(grammar_file, 'r', encoding='utf8'))
            grammar_file = os.path.join(str(tmpdir), 'grammar.lt')
            codecs.open(grammar_file, 'w', encoding='utf8').write(
                '\n'.join(grammar_unicode))
        segmented = list(algos[algo].segment(

          prepared_text, grammar_file, 'Colloc0', nruns=1))

../test/test_pipeline.py:77:

---

../wordseg/algos/ag.py:719: in segment
for n in range(nruns))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u10da \u0b16 r', 'dh \u0807 r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d \u0b16 d y uw iy t ah p', 'uw p s', 'l \u0b16 t ax l b \u0b16 t m ao r', ...]
grammar_file = '/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_pipeline_ag_unicode_0/grammar.lt'
args = '-E -d 0 -a 0.0001 -b 10000 -e 1 -f 1 -g 100 -h 0.01 -R -1 -P -x 10 -r 37059'
test_text = '\u10da \u0b16 r\ndh \u0807 r w iy g ow\ns ow ax m p ey sh ax n t\nw ah t d \u0b16 d y uw iy t ah p\nuw p s\nl \u0b16 t ax l b \u0b16 t m ao r\nl \u0b16 t ax l...iy\nuw p s iy b \u0b16 t iy b \u0b16 t iy b \u0b16 t iy b ey b iy g er l\ny uw \u10da aa s p ae g \u10da \u0807 t iy ow z\nw aa n t s ax m m ao r\nn ow\n'
log_level = 30, log_name = 'wordseg-ag - run 1'

def _run_ag_single(text, grammar_file, args, test_text=None,
                   log_level=logging.ERROR, log_name='wordseg-ag'):
    """Runs the AG program a single time and returns the computed parse trees

    Parameters
    ----------
    text : sequence
        The list of utterances to train the model on, and to segment
        if `test_text` is None.
    grammar_file : str
        The path to the grammar file to use for segmentation
    args : str
        Command line options to run the AG program with, use
        'wordseg-ag --help' to have a complete list of available
        options
    test_text : sequence, optional
        If not None, the list of utterances to segment on the model
        learned from `text`
    log : logging.Logger, optional
        A logger where to send log messages

    Returns
    -------
    A list of parse trees, one tree per input utterance in `text`

    Raises
    ------
    RuntimeError
        If the AG program fails and returns an error code

    """
    log = utils.get_logger(name=log_name, level=log_level)

    # setup the train text as a multiline string
    train_text = '\n'.join(utt.strip() for utt in text) + '\n'

    # setup the test text as well
    test_text = train_text if test_text is None else (
        '\n'.join(utt.strip() for utt in test_text) + '\n')

    # we need to write the test text as a tempfile, so we create a
    # tempdir. The directory and its content is automatically erased
    # when done
    temp_dir = tempfile.mkdtemp()
    try:
        # write the test text as a temporary file. ylt extension is
        # the one used in the original AG implementation
        test_tmpfile = os.path.join(temp_dir, 'test.ylt')
        codecs.open(test_tmpfile, 'w', encoding='utf8').write(test_text)

        # generate the command to run as a subprocess
        command = ('{binary} {grammar} {args} -u {test} '.format(
            binary=utils.get_binary('ag'),
            grammar=grammar_file,
            args=args,
            test=test_tmpfile))

        log.info('running %s', command)

        # run the command as a subprocess
        process = subprocess.Popen(
            shlex.split(command),
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        parses, messages = process.communicate(train_text.encode('utf8'))

        # log.debug the AG messages AFTER EXECUTION
        messages = messages.decode('utf8').split('\n')
        for msg in messages:
            log.debug(re.sub('^# ', '', msg).strip())

        # fail if AG returns an error code
        if process.returncode:
            raise RuntimeError(

              'fails with error code {}'.format(process.returncode))

E RuntimeError: fails with error code -6

../wordseg/algos/ag.py:447: RuntimeError
__________________________ test_pipeline[dpseg-ascii] __________________________

algo = 'dpseg', encoding = 'ascii'
tags = ['hh ih r ;esyll ;eword', 'dh eh r ;esyll ;eword w iy ;esyll ;eword g ow ;esyll ;eword', 's ow ;esyll ;eword ax m p ;e...yll ;eword', 'uw p s ;esyll ;eword', 'l ih t ;esyll ax l ;esyll ;eword b ih t ;esyll ;eword m ao r ;esyll ;eword', ...]
tmpdir = local('/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_pipeline_dpseg_ascii_0')

@pytest.mark.parametrize('algo, encoding', params)
def test_pipeline(algo, encoding, tags, tmpdir):
    # the token separator we use in the whole pipeline
    separator = Separator(phone=' ', syllable=';esyll', word=';eword')

    # add some unicode chars in the input text
    if encoding == 'unicode':
        tags = add_unicode(tags)

    # build the gold version from the tags
    gold = list(wordseg.prepare.gold(tags, separator=separator))
    assert len(gold) == len(tags)
    for a, b in zip(gold, tags):
        assert separator.remove(a) == separator.remove(b)

    # prepare the text for segmentation
    prepared_text = list(wordseg.prepare.prepare(tags, separator=separator))
    assert len(prepared_text) == len(tags)
    for a, b in zip(prepared_text, tags):
        assert separator.remove(a) == separator.remove(b)

    # segment it with the given algo (use default options)
    if algo in ('dpseg', 'puddle'):
        # only 1 fold for iterative algos: faster

      segmented = list(algos[algo].segment(prepared_text, nfolds=1))

../test/test_pipeline.py:63:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc6\u0bbf\u0bc3', '\u0bc4\u0bbc\u0bc3\u0bd2\u0bb9\u0bc9\u0bd3', '\u0bcf\u0bd3\u0bc1\u0bcb\u0bce\u0bc0\u0bd0\u0bc1\u0bcd\u0bd1', '\u0bd2\u0bbd\u0bd1\u0bc8\u0bbf\u0bc8\u0bc5\u0bca\u0bb9\u0bd1\u0bbd\u0bce', '\u0bca\u0bce\u0bcf', '\u0bcc\u0bbf\u0bd1\u0bc1\u0bcc\u0bc7\u0bbf\u0bd1\u0bcb\u0bbe\u0bc3', ...]
args = '--ngram 1 --a1 0 --b1 1', log_level = 30
log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
_________________________ test_pipeline[dpseg-unicode] _________________________

algo = 'dpseg', encoding = 'unicode'
tags = ['\u10da \u0b16 r ;esyll ;eword', 'dh \u0807 r ;esyll ;eword w iy ;esyll ;eword g ow ;esyll ;eword', 's ow ;esyll ;eword ax m p ;esyl...esyll ;eword', 'uw p s ;esyll ;eword', 'l \u0b16 t ;esyll ax l ;esyll ;eword b \u0b16 t ;esyll ;eword m ao r ;esyll ;eword', ...]
tmpdir = local('/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_pipeline_dpseg_unicode_0')

@pytest.mark.parametrize('algo, encoding', params)
def test_pipeline(algo, encoding, tags, tmpdir):
    # the token separator we use in the whole pipeline
    separator = Separator(phone=' ', syllable=';esyll', word=';eword')

    # add some unicode chars in the input text
    if encoding == 'unicode':
        tags = add_unicode(tags)

    # build the gold version from the tags
    gold = list(wordseg.prepare.gold(tags, separator=separator))
    assert len(gold) == len(tags)
    for a, b in zip(gold, tags):
        assert separator.remove(a) == separator.remove(b)

    # prepare the text for segmentation
    prepared_text = list(wordseg.prepare.prepare(tags, separator=separator))
    assert len(prepared_text) == len(tags)
    for a, b in zip(prepared_text, tags):
        assert separator.remove(a) == separator.remove(b)

    # segment it with the given algo (use default options)
    if algo in ('dpseg', 'puddle'):
        # only 1 fold for iterative algos: faster

      segmented = list(algos[algo].segment(prepared_text, nfolds=1))

../test/test_pipeline.py:63:

---

../wordseg/algos/dpseg.py:325: in segment
for n, fold in enumerate(folded_texts))
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:779: in call
while self.dispatch_one_batch(iterator):
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py:332: in init
self.results = batch()
/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py:131: in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]

---

text = ['\u0bc6\u0bc0\u0bc3', '\u0bc4\u0bbc\u0bc3\u0bd2\u0bb9\u0bc9\u0bd3', '\u0bcf\u0bd3\u0bc1\u0bcb\u0bce\u0bbf\u0bd0\u0bc1\u0bcd\u0bd1', '\u0bd2\u0bbd\u0bd1\u0bc8\u0bc0\u0bc8\u0bc5\u0bca\u0bb9\u0bd1\u0bbd\u0bce', '\u0bca\u0bce\u0bcf', '\u0bcc\u0bc0\u0bd1\u0bc1\u0bcc\u0bc7\u0bc0\u0bd1\u0bcb\u0bbe\u0bc3', ...]
args = '--ngram 1 --a1 0 --b1 1', log_level = 30
log_name = 'wordseg-dpseg - fold 1'

def _dpseg(text, args, log_level=logging.ERROR, log_name='wordseg-dpseg'):
    log = utils.get_logger(name=log_name, level=log_level)

    with tempfile.NamedTemporaryFile() as tmp_output:
        command = '{binary} --output-file {output} {args}'.format(
            binary=utils.get_binary('dpseg'),
            output=tmp_output.name,
            args=args)

        log.debug('running "%s"', command)

        process = subprocess.Popen(
            shlex.split(command), stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        def writer():
            for utt in text:
                process.stdin.write((utt.strip() + '\n').encode('utf8'))
            process.stdin.close()

        thread = threading.Thread(target=writer)
        thread.start()

        # Send stdout and stderr to logger, break if EOF reached
        while True:
            line_out = process.stdout.readline().decode('utf8')
            line_err = process.stderr.readline().decode('utf8')

            if line_out == "" and line_err == "":
                break

            if line_out != "":
                log.debug(line_out.strip())

            if line_err != "":
                log.error(line_err.strip())

        thread.join()
        process.wait()
        if process.returncode:
            raise RuntimeError(

              'failed with error code {}'.format(process.returncode))

E RuntimeError: failed with error code -11

../wordseg/algos/dpseg.py:293: RuntimeError
______________________________ test_tutorial[py] _______________________________

tags = ['hh ih r ;esyll ;eword', 'dh eh r ;esyll ;eword w iy ;esyll ;eword g ow ;esyll ;eword', 's ow ;esyll ;eword ax m p ;e...yll ;eword', 'uw p s ;esyll ;eword', 'l ih t ;esyll ax l ;esyll ;eword b ih t ;esyll ;eword m ao r ;esyll ;eword', ...]
tmpdir = local('/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_tutorial_py_0')
ext = 'py'

@pytest.mark.parametrize('ext', ['py', 'sh'])
def test_tutorial(tags, tmpdir, ext):
    tutorial_dir = os.path.abspath(
        os.path.join(os.path.dirname(__file__), '..', 'doc'))
    assert os.path.isdir(tutorial_dir)

    script = os.path.join(tutorial_dir, 'tutorial.' + ext)
    assert os.path.isfile(script)

    p = os.path.join(str(tmpdir), 'input.txt')
    codecs.open(p, 'w', encoding='utf8').write('\n'.join(tags) + '\n')

  subprocess.check_call([script, p], cwd=str(tmpdir))

../test/test_tutorial.py:23:

---

popenargs = (['/Users/acristia/Documents/wordseg_utest/doc/tutorial.py', '/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_tutorial_py_0/input.txt'],)
kwargs = {'cwd': '/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_tutorial_py_0'}
retcode = 1
cmd = ['/Users/acristia/Documents/wordseg_utest/doc/tutorial.py', '/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_tutorial_py_0/input.txt']

def check_call(*popenargs, **kwargs):
    """Run command with arguments.  Wait for command to complete.  If
    the exit code was zero then return, otherwise raise
    CalledProcessError.  The CalledProcessError object will have the
    return code in the returncode attribute.

    The arguments are the same as for the Popen constructor.  Example:

    check_call(["ls", "-l"])
    """
    retcode = call(*popenargs, **kwargs)
    if retcode:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]

      raise CalledProcessError(retcode, cmd)

E CalledProcessError: Command '['/Users/acristia/Documents/wordseg_utest/doc/tutorial.py', '/private/var/folders/mx/f_gpzmq105x8sgqfdtbr4zrr0000gn/T/pytest-of-acristia/pytest-10/test_tutorial_py_0/input.txt']' returned non-zero exit status 1

/anaconda/lib/python2.7/subprocess.py:186: CalledProcessError
----------------------------- Captured stdout call -----------------------------

• Statistics

{
"phones": {
"tokens": 170,
"hapaxes": 5,
"types": 29
},
"corpus": {
"nutts_single_word": 4,
"nutts": 13,
"entropy": 0.044723035155421365,
"mattr": 0.7166666666666667
},
"words": {
"tokens": 34,
"hapaxes": 19,
"types": 24
}
}
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
File "/Users/acristia/Documents/wordseg_utest/doc/tutorial.py", line 37, in
segmented_dpseg = dpseg.segment(prepared, nfolds=1)
File "build/bdist.macosx-10.7-x86_64/egg/wordseg/algos/dpseg.py", line 325, in segment
File "/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py", line 779, in call
while self.dispatch_one_batch(iterator):
File "/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py", line 625, in dispatch_one_batch
self._dispatch(tasks)
File "/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py", line 588, in _dispatch
job = self._backend.apply_async(batch, callback=cb)
File "/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py", line 111, in apply_async
result = ImmediateResult(func)
File "/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/_parallel_backends.py", line 332, in init
self.results = batch()
File "/anaconda/lib/python2.7/site-packages/joblib-0.11-py2.7.egg/joblib/parallel.py", line 131, in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "build/bdist.macosx-10.7-x86_64/egg/wordseg/algos/dpseg.py", line 293, in _dpseg
RuntimeError: failed with error code -11
==================== 24 failed, 171 passed in 44.43 seconds ====================

[mmmaat]

Ok, the dpseg is still not working on MacOS (see #4), but AG should be ok.

Can you please report the output of pytest -vx test/test_algo_ag.py::test_ag_single_run for AG ?

Thanks!

[alecristia]

​ Alejandrinas-MacBook-Air:wordseg_utest acristia$ pytest -vx test/test_algo_ag.py::test_ag_single_ru *============================= test session starts ==============================* platform darwin -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /anaconda/bin/python cachedir: .cache rootdir: /Users/acristia/Documents/wordseg_utest, inifile: *collecting 9 items* *========================= no tests ran in 1.28 seconds =========================* ERROR: not found: /Users/acristia/Documents/wordseg_utest/test/test_algo_ag.py::test_ag_single_ru (no name '/Users/acristia/Documents/wordseg_utest/test/test_algo_ag.py::test_ag_single_ru' in any of [<Module 'test/test_algo_ag.py'>]) Alejandrinas-MacBook-Air:wordseg_utest acristia$ cd test/ Alejandrinas-MacBook-Air:test acristia$ pytest -vx test/test_algo_ag.py::test_ag_single_ru *======================================================================= test session starts ========================================================================* platform darwin -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /anaconda/bin/python cachedir: .cache rootdir: /Users/acristia/Documents/wordseg_utest/test, inifile: *=================================================================== no tests ran in 0.00 seconds ===================================================================* ERROR: file not found: test/test_algo_ag.py::test_ag_single_ru Alejandrinas-MacBook-Air:test acristia$ ls

…

__init__.py test_algo_ag.py test_algo_puddle.py test_pipeline.py test_syllabification.py

__init__.pyc test_algo_baseline.py test_algo_tp.py test_prepare.py test_tutorial.py

__pycache__ test_algo_dibs.py test_evaluate.py test_separator.py test_utils.py data test_algo_dpseg.py test_folding.py test_statistics.py test_wordseg_commands.py Alejandrinas-MacBook-Air:test acristia$ cd .. Alejandrinas-MacBook-Air:wordseg_utest acristia$ git pull remote: Counting objects: 28, done. remote: Compressing objects: 100% (12/12), done. remote: Total 28 (delta 18), reused 26 (delta 16), pack-reused 0 Unpacking objects: 100% (28/28), done. From https://github.com/bootphon/wordseg 74b99fd..65aa776 master -> origin/master 74b99fd..711242e new-docs -> origin/new-docs ec7e97d..9d92f4e readthedocs-pages -> origin/readthedocs-pages Updating 74b99fd..65aa776 Fast-forward doc/tutorial.sh | 2 +- wordseg/evaluate.py | 13 +++++++------ 2 files changed, 8 insertions(+), 7 deletions(-) Alejandrinas-MacBook-Air:wordseg_utest acristia$ pytest -vx test/test_algo_ag.py::test_ag_single_ru *======================================================================= test session starts ========================================================================* platform darwin -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /anaconda/bin/python cachedir: .cache rootdir: /Users/acristia/Documents/wordseg_utest, inifile: *collecting 9 items* *=================================================================== no tests ran in 0.73 seconds ===================================================================* ERROR: not found: /Users/acristia/Documents/wordseg_utest/test/test_algo_ag.py::test_ag_single_ru (no name '/Users/acristia/Documents/wordseg_utest/test/test_algo_ag.py::test_ag_single_ru' in any of [<Module 'test/test_algo_ag.py'>]) Alejandrinas-MacBook-Air:wordseg_utest acristia$ cd test/ Alejandrinas-MacBook-Air:test acristia$ pytest -vx test/test_algo_ag.py::test_ag_single_ru *======================================================================= test session starts ========================================================================* platform darwin -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /anaconda/bin/python cachedir: .cache rootdir: /Users/acristia/Documents/wordseg_utest/test, inifile: *=================================================================== no tests ran in 0.00 seconds ===================================================================* ERROR: file not found: test/test_algo_ag.py::test_ag_single_ru

[mmmaat]

test/test_algo_ag.py::test_ag_single_run and not test/test_algo_ag.py::test_ag_single_ru ;)

[alecristia]

Ooops!

Results below; is it normal to have the line breaks: test_text = 'hh ih r**\ndh eh r w iy g ow\ns ow ax m p ey sh ax n t\n** ... '?

============================= test session starts ==============================
platform darwin -- Python 3.6.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0 -- /anaconda/envs/wordseg/bin/python
cachedir: .cache
rootdir: /Users/alejandrinacristia/gitrepos/wordseg_utest, inifile:
collecting ... collected 1 item

test/test_algo_ag.py::test_ag_single_run FAILED

=================================== FAILURES ===================================
______________________________ test_ag_single_run ______________________________

prep = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]

def test_ag_single_run(prep):
    raw_parses = ag._run_ag_single(

      prep, os.path.join(grammar_dir, grammars[1][0]), args=test_arguments)

test/test_algo_ag.py:31:

---

text = ['hh ih r', 'dh eh r w iy g ow', 's ow ax m p ey sh ax n t', 'w ah t d ih d y uw iy t ah p', 'uw p s', 'l ih t ax l b ih t m ao r', ...]
grammar_file = '/Users/alejandrinacristia/Library/Caches/Python-Eggs/wordseg-0.6.1-py3.6.egg-tmp/data/ag/Colloc0_enFestival.lt'
args = '-E -P -R -1 -n 10 -a 0.0001 -b 10000.0 -e 1.0 -f 1.0 -g 100.0 -h 0.01 -x 2'
test_text = 'hh ih r\ndh eh r w iy g ow\ns ow ax m p ey sh ax n t\nw ah t d ih d y uw iy t ah p\nuw p s\nl ih t ax l b ih t m ao r...iy b ih t iy b ih t iy b ih t iy b ey b iy g er l\ny uw hh aa s p ae g hh eh t iy ow z\nw aa n t s ax m m ao r\nn ow\n'
log_level = 40, log_name = 'wordseg-ag'

def _run_ag_single(text, grammar_file, args, test_text=None,
                   log_level=logging.ERROR, log_name='wordseg-ag'):
    """Runs the AG program a single time and returns the computed parse trees

    Parameters
    ----------
    text : sequence
        The list of utterances to train the model on, and to segment
        if `test_text` is None.
    grammar_file : str
        The path to the grammar file to use for segmentation
    args : str
        Command line options to run the AG program with, use
        'wordseg-ag --help' to have a complete list of available
        options
    test_text : sequence, optional
        If not None, the list of utterances to segment on the model
        learned from `text`
    log : logging.Logger, optional
        A logger where to send log messages

    Returns
    -------
    A list of parse trees, one tree per input utterance in `text`

    Raises
    ------
    RuntimeError
        If the AG program fails and returns an error code

    """
    log = utils.get_logger(name=log_name, level=log_level)

    # setup the train text as a multiline string
    train_text = '\n'.join(utt.strip() for utt in text) + '\n'

    # setup the test text as well
    test_text = train_text if test_text is None else (
        '\n'.join(utt.strip() for utt in test_text) + '\n')

    # we need to write the test text as a tempfile, so we create a
    # tempdir. The directory and its content is automatically erased
    # when done
    temp_dir = tempfile.mkdtemp()
    try:
        # write the test text as a temporary file. ylt extension is
        # the one used in the original AG implementation
        test_tmpfile = os.path.join(temp_dir, 'test.ylt')
        codecs.open(test_tmpfile, 'w', encoding='utf8').write(test_text)

        # generate the command to run as a subprocess
        command = ('{binary} {grammar} {args} -u {test} '.format(
            binary=utils.get_binary('ag'),
            grammar=grammar_file,
            args=args,
            test=test_tmpfile))

        log.info('running %s', command)

        # run the command as a subprocess
        process = subprocess.Popen(
            shlex.split(command),
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        parses, messages = process.communicate(train_text.encode('utf8'))

        # log.debug the AG messages AFTER EXECUTION
        messages = messages.decode('utf8').split('\n')
        for msg in messages:
            log.debug(re.sub('^# ', '', msg).strip())

        # fail if AG returns an error code
        if process.returncode:
            raise RuntimeError(

              'fails with error code {}'.format(process.returncode))

E RuntimeError: fails with error code -6

wordseg/algos/ag.py:447: RuntimeError
!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!
=========================== 1 failed in 0.36 seconds ===========================

PS: On another mac now, can replicate the original error (although I see tests are MUCH faster here! interesting...)

[mmmaat]

At this level this is normal to have line breaks as the text is wrote in a temp file: codecs.open(test_tmpfile, 'w', encoding='utf8').write(test_text). But I see that the docstring is not up to date.

For your error, I definitely need to work on a mac to understand what appends... (for dpseg as well)

[mmmaat]

From Mac, dpseg and ag can now be used from docker. We do not plan to support it directly.