Contradictions in text

Contradictions in text: Been conducting some research on natural language processing, blending textual entailment using probabilistic language models with graph based associative text extraction.

Take the two sentences:

1. Good oil shows and good poroperm values were observed in core x in the Raptor Sandstone.

2. The Raptor Sandstone was water wet.

The predictive model returns the probability that these 2 sentences contradict each other as follows:

Entailment: 0.2%

Neutral: 1.2%

Contradiction: 98.6%

Some promising early results to help scientists and analysts zero in on inconsistencies and contradictions across documents that they may not have otherwise been aware of.

Contradictions are always fertile ground for new theories and investigations. Having automated agents to help collate candidates from vast amounts of text, could have significant value.

A key aspect is likely to be using long span co-reference and ‘choosing’ the right type of sentences to compare. Otherwise algorithms may indicate a ‘technical’ contradiction, but in the wider context this will generate far too many false positives from a real world perspective.

More research to be conducted!