graviti
Products
Resources
About us
Children Book Test
Text
NLP
|...
License: Unknown

Overview

Children’s Book Test (CBT), designed to measure directly how well language models can exploit
wider linguistic context. The CBT is built from books that are freely available thanks to Project
Gutenberg
. Details and baseline results on this dataset can be
found in the paper:

Felix Hill, Antoine Bordes, Sumit Chopra and Jason Weston. The Goldilocks
Principle: Reading Children’s Books with Explicit Memory Representations
,
arXiv:1511.02301.

After allocating books to either training, validation or test sets, we formed
example ‘questions’ from chapters in the book by enumerating 21 consecutive sentences. In each
question, the first 20 sentences form the context, and a word is removed from the 21st sentence,
which becomes the query. Models must identify the answer word among a selection of 10 candidate
answers appearing in the context sentences and the query. For finer-grained analyses, we evaluated
four classes of question by removing distinct types of word: Named Entities, (Common) Nouns,
Verbs and Prepositions

Here is an example of question (context + query) from Alice in Wonderland by Lewis Carroll:

Context:
1 So they had to fall a long way .
2 So they got their tails fast in their mouths .
3 So they could n't get them out again .
4 That 's all .
5 `` Thank you , " said Alice , `` it 's very interesting .
6 I never knew so much about a whiting before . "
7 `` I can tell you more than that , if you like , " said the Gryphon .
8 `` Do you know why it 's called a whiting ? "
9 `` I never thought about it , " said Alice .
10 `` Why ? "
11 `` IT DOES THE BOOTS AND SHOES . '
12 the Gryphon replied very solemnly .
13 Alice was thoroughly puzzled .
14 `` Does the boots and shoes ! "
15 she repeated in a wondering tone .
16 `` Why , what are YOUR shoes done with ? "
17 said the Gryphon .
18 `` I mean , what makes them so shiny ? "
19 Alice looked down at them , and considered a little before she gave her answer .
20 `` They 're done with blacking , I believe . "

Query: `` Boots and shoes under the sea , " the XXXXX went on in a
deep voice , `` are done with a whiting ".

Candidates: Alice|BOOTS|Gryphon|SHOES|answer|fall|mouths|tone|way|whiting

Answer: gryphon

Citation

Please use the following citation when referencing the dataset:

@article{hill2015goldilocks,
  title={The goldilocks principle: Reading children's books with explicit memory representations},
  author={Hill, Felix and Bordes, Antoine and Chopra, Sumit and Weston, Jason},
  journal={arXiv preprint arXiv:1511.02301},
  year={2015}
}
Data Summary
Type
Text,
Amount
--
Size
114.96MB
Provided by
Facebook Research
Giving people the power to build community through research and innovation
| Amount -- | Size 114.96MB
Children Book Test
Text
NLP
License: Unknown

Overview

Children’s Book Test (CBT), designed to measure directly how well language models can exploit
wider linguistic context. The CBT is built from books that are freely available thanks to Project
Gutenberg
. Details and baseline results on this dataset can be
found in the paper:

Felix Hill, Antoine Bordes, Sumit Chopra and Jason Weston. The Goldilocks
Principle: Reading Children’s Books with Explicit Memory Representations
,
arXiv:1511.02301.

After allocating books to either training, validation or test sets, we formed
example ‘questions’ from chapters in the book by enumerating 21 consecutive sentences. In each
question, the first 20 sentences form the context, and a word is removed from the 21st sentence,
which becomes the query. Models must identify the answer word among a selection of 10 candidate
answers appearing in the context sentences and the query. For finer-grained analyses, we evaluated
four classes of question by removing distinct types of word: Named Entities, (Common) Nouns,
Verbs and Prepositions

Here is an example of question (context + query) from Alice in Wonderland by Lewis Carroll:

Context:
1 So they had to fall a long way .
2 So they got their tails fast in their mouths .
3 So they could n't get them out again .
4 That 's all .
5 `` Thank you , " said Alice , `` it 's very interesting .
6 I never knew so much about a whiting before . "
7 `` I can tell you more than that , if you like , " said the Gryphon .
8 `` Do you know why it 's called a whiting ? "
9 `` I never thought about it , " said Alice .
10 `` Why ? "
11 `` IT DOES THE BOOTS AND SHOES . '
12 the Gryphon replied very solemnly .
13 Alice was thoroughly puzzled .
14 `` Does the boots and shoes ! "
15 she repeated in a wondering tone .
16 `` Why , what are YOUR shoes done with ? "
17 said the Gryphon .
18 `` I mean , what makes them so shiny ? "
19 Alice looked down at them , and considered a little before she gave her answer .
20 `` They 're done with blacking , I believe . "

Query: `` Boots and shoes under the sea , " the XXXXX went on in a
deep voice , `` are done with a whiting ".

Candidates: Alice|BOOTS|Gryphon|SHOES|answer|fall|mouths|tone|way|whiting

Answer: gryphon

Citation

Please use the following citation when referencing the dataset:

@article{hill2015goldilocks,
  title={The goldilocks principle: Reading children's books with explicit memory representations},
  author={Hill, Felix and Bordes, Antoine and Chopra, Sumit and Weston, Jason},
  journal={arXiv preprint arXiv:1511.02301},
  year={2015}
}
0
Start building your AI now
graviti
wechat-QR
Long pressing the QR code to follow wechat official account

Copyright@Graviti