graviti
Products
Resources
About us
clincOos
Text
NLP
|...
License: CC BY 3.0

Overview

Task-oriented dialog systems need to know when a query falls outside their range of supported
intents, but current text classification corpora only define label sets that cover every example.
We introduce a new dataset that includes queries that are out-of-scope (OOS), i.e., queries
that do not fall into any of the system's supported intents. This poses a new challenge because
models cannot assume that every query at inference time belongs to a system-supported intent
class. Our dataset also covers 150 intent classes over 10 domains, capturing the breadth that
a production task-oriented agent must handle. It offers a way of more rigorously and realistically
benchmarking text classification in task-driven dialog systems.

Data Format

FeaturesDict({
    'domain': tf.int32,
    'domain_name': Text(shape=(), dtype=tf.string),
    'intent': tf.int32,
    'intent_name': Text(shape=(), dtype=tf.string),
    'text': Text(shape=(), dtype=tf.string),
})

Citation

If you find our dataset useful, please be sure to cite:

@inproceedings{larson-etal-2019-evaluation,
    title = "An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction",
    author = "Larson, Stefan  and
      Mahendran, Anish  and
      Peper, Joseph J.  and
      Clarke, Christopher  and
      Lee, Andrew  and
      Hill, Parker  and
      Kummerfeld, Jonathan K.  and
      Leach, Kevin  and
      Laurenzano, Michael A.  and
      Tang, Lingjia  and
      Mars, Jason",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language
Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)",
    year = "2019",
    url = "https://www.aclweb.org/anthology/D19-1131"
}

License

CC BY 3.0

Data Summary
Type
Text,
Amount
23.7K
Size
2.5MB
Provided by
Clinc, Inc.
Clinc is the world leader in conversational AI research and its application for the enterprise. We're on a mission to push the boundaries of conversational AI, empowering enterprises to deliver a new and revolutionary AI experience for their customers.
| Amount 23.7K | Size 2.5MB
clincOos
Text
NLP
License: CC BY 3.0

Overview

Task-oriented dialog systems need to know when a query falls outside their range of supported
intents, but current text classification corpora only define label sets that cover every example.
We introduce a new dataset that includes queries that are out-of-scope (OOS), i.e., queries
that do not fall into any of the system's supported intents. This poses a new challenge because
models cannot assume that every query at inference time belongs to a system-supported intent
class. Our dataset also covers 150 intent classes over 10 domains, capturing the breadth that
a production task-oriented agent must handle. It offers a way of more rigorously and realistically
benchmarking text classification in task-driven dialog systems.

Data Format

FeaturesDict({
    'domain': tf.int32,
    'domain_name': Text(shape=(), dtype=tf.string),
    'intent': tf.int32,
    'intent_name': Text(shape=(), dtype=tf.string),
    'text': Text(shape=(), dtype=tf.string),
})

Citation

If you find our dataset useful, please be sure to cite:

@inproceedings{larson-etal-2019-evaluation,
    title = "An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction",
    author = "Larson, Stefan  and
      Mahendran, Anish  and
      Peper, Joseph J.  and
      Clarke, Christopher  and
      Lee, Andrew  and
      Hill, Parker  and
      Kummerfeld, Jonathan K.  and
      Leach, Kevin  and
      Laurenzano, Michael A.  and
      Tang, Lingjia  and
      Mars, Jason",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language
Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)",
    year = "2019",
    url = "https://www.aclweb.org/anthology/D19-1131"
}

License

CC BY 3.0

0
Start building your AI now
graviti
wechat-QR
Long pressing the QR code to follow wechat official account

Copyright@Graviti