Natural Language Processing (Steven Bird)
NLTK, linguistics. Together with 'Natural Language Processing with Python' by Steven Bird.
49 flashcards · Computer Science · by lakesare
What's plural for corpus?
Corpora. A corpus (plural corpora) or text corpus is a language resource consisting of a large and structured set of texts (nowadays usually electronically stored and processed).
What are these corpora categories? Monolingual Comparable Parallel A corpus is generally expected to have additional characteristics: corpora are usually constructed so as to be balanced and representative of a particular domain (McEnery and Wilson, 2001, pp. 29–30). (Sometimes the term is used more loosely to cover any large collection of language data which need not have been compiled systematically, as in the phrase ‘the web as corpus’.)
Monolingual corpora - texts from a single language. Monolingual corpora consist, as the name suggests, of data from a single language.Comparable corpora - texts from many languages. Comparable compora include a range of monolingual corpora in different languages, preferably with a similar level of balance and representativeness, and can be used for contrastive studies of those languages.Parallel corpora - texts with translations. Parallel corpora include original texts in one language with tran…
What's meant by reference VS monitor corpora?
reference corpora - remain fixed once compiled. Linguists who do (computational) experiments like reference corpora, whose composition is fixed once and for all, and which are publicly available. This lets them compare their results with those of other workers in a meaningful way. For example, P.F. Brown's group suggest using the cross-entropy of a language model measured on the Brown corpus as a benchmark for language modelling.monitor corpora - continually updated with new material. Lexicogra…
What's transitive VS intransitive verb?
transitive verb - demands an object (Please bring coffee). A transitive verb is a verb that requires an object to receive the action.intransitive verb - fine on its own (Susan laughed). An intransitive verb does not take an object. Using an object immediately after an intransitive verb will create an incorrect sentence. A transitive verb needs to transfer its action to something or someone - an object. In essence, transitive means "to affect something else".
What's a treebank?
Treebank is a parsed text corpus that annotates syntactic or semantic sentence structure. The exploitation of treebank data has been important ever since the first large-scale treebank, The Penn Treebank, was published. However, although originating in computational linguistics, the value of treebanks is becoming more widely appreciated in linguistics research as a whole. Most syntactic treebanks annotate variants of either phrase structure (left) or dependency structure (right).
What's synset?
synset (plural synsets) (information science) A set of one or more synonyms that are interchangeable in some context without changing the truth value of the proposition in which they are embedded.
Name 2 prominent text corpora.
Penn Treebank - a parsed text corpus that annotates syntactic or semantic sentence structure. The Penn Treebank with its various offshoots is one of the widely used linguistic resources among empirical researchers. It includes a collection of texts in four different formats: Raw text (original). Tagged with POS (parts of speech) using a tagset which was developed as part of the project. ‘Parsed’; that is, marked up with constituent structure. Combined, including both POS tags and constituent st…
How does hapax legomenon translate from Ancient Greek?
hápax - once legómenon - said In corpus linguistics, a hapax legomenon, (pl. hapax legomena; sometimes abbreviated to hapax) is a word or an expression that occurs only once within some text.
What's a hapax legomenon (or just hapax)?
A word or expression that occurs only once within some text. In corpus linguistics, a hapax legomenon, (pl. hapax legomena; sometimes abbreviated to hapax) is a word or an expression that occurs only once within some text.From the Ancient Greek: hápax (once) legómenon (said)
How would you recreate this set syntax in python? { w | w ∈ V & P(w) }
[w for w in V if p(w)] This is called a list comprehension. Note that it produces a list, not a set, which means that duplicates are possible.) Observe how similar the two notations are.
What does the nltk.bigrams function do?
In the fields of computational linguistics and probability, an n-gram is a contiguous sequence of n items from a given sample of text or speech. The items can be phonemes, syllables, letters, words or base pairs according to the application.
What does the nltk.trigrams function do?
In the fields of computational linguistics and probability, an n-gram is a contiguous sequence of n items from a given sample of text or speech. The items can be phonemes, syllables, letters, words or base pairs according to the application.
What's a concordance? Some simple techniques for analysing corpora include- concordancing,- collocations, and- (conditional) frequency distributions.
Concordance is every occurrence of a word, together with some context. A concordance is an alphabetical list of the principal words used in a book or body of work, listing every instance of each word with its immediate context. A concordance view shows us every occurrence of a given word, together with some context. Here we look up the word monstrous in Moby Dick. Concordances have been compiled only for works of special importance, such as the Vedas, Bible, Quran or the works of Shakespeare, b…
What's a collocation? Some simple techniques for analysing corpora include- concordancing,- collocations, and- (conditional) frequency distributions.
Collocations are series of words that occur together unusually often (more frequently than expected by chance). In corpus linguistics, a collocation is a series of words or terms that co-occur more often than would be expected by chance.
What method is used to calculate conditional frequency distribution in nltk? Some simple techniques for analysing corpora include- concordancing,- collocations, and- (conditional) frequency distributions.
FreqDist >>> import nltk >>> from nltk.probability import FreqDist >>> >>> FreqDist(text1) Conditional frequency distributions support an elementary form of statistical analysis.A frequency distribution counts observable events and a conditional frequency distribution pairs each event with a condition. Some sample applications are:- Comparing the use of particular words in different genres.- Comparing word lengths in different languages.
How are collocations connected to bigrams?
Collocations are essentially just frequent bigrams, except that we want to pay more attention to the cases that involve rare words. In particular, we want to find bigrams that occur more often than we would expect based on the frequency of individual words. Collocations are series of words that occur together unusually often (more frequently than expected by chance).
What's alphanumeric string?
Alphanumeric string consists of both alphabetical and numerical characters.
What's the Recognizing Textual Entailment (RTE) task?
We're given the text & the hypothesis - and we have to say whether the hypothesis is right based on our text. Text: David Golinkin is the editor or author of 18 books, and over 150 responsa, articles, sermons and books b.Hypothesis: Golinkin has written 18 books.In order to determine whether the hypothesis is supported by the text, the system needs the following background knowledge: (i) if someone is an author of a book, then he/ she has written that book; (ii) if someone is an editor of a boo…
Describe these common structures of text corpora: isolated categorized overlapping temporal
isolated - isolated texts with no particular organization The simplest kind of corpus is a collection of isolated texts with no particular organization.categorized - non-overlapping categories Some corpora are structured into categories, such as genre (Brown Corpus).overlapping - overlapping categories Some categorizations overlap, such as topic categories (Reuters Corpus).temporal - organized over time Other corpora represent language use over time (Inaugural Address Corpus).
What are homonyms?
What are stop words in linguistics?
Words that we want to filter out before processing our natural language. Usually stop words are high-frequency words like (the, to, and also). Stopwords usually have little lexical content, and their presence in a text fails to distinguish it from other texts. In computing, stop words are words which are filtered out before or after processing of natural language data (text). Though "stop words" usually refers to the most common words in a language, there is no single universal list of stop wor…
What's a Swadesh list?
List of about 200 common words in several languages. The Swadesh list was put together by Morris Swadesh on the basis of his intuitions. The Swadesh list is a classic compilation of basic concepts for the purposes of historical-comparative linguistics. Translations of the Swadesh list into a set of languages allow researchers to quantify the interrelatedness of those languages. The languages are identified using an ISO 639 two-letter code
What would swadesh.entries(['fr', 'en']) return?
Cognate words.
What's a thesaurus?
A thesaurus is a synonym dictionary. It's used for finding synonyms (and sometimes antonyms) of words. Thesauri (or thesauruses) are often used by writers to help find the best word to express an idea. The word "thesaurus" comes from Latin thēsaurus, which in turn comes from Greek θησαυρός (thēsauros) 'treasure, treasury, storehouse'. The word thēsauros is of uncertain etymology.Until the 19th century, a thesaurus was any dictionary or encyclopedia, as in the Thesaurus Linguae Latinae (Dictiona…
What's in the WordNet?
WordNet can thus be seen as a combination and extension of a dictionary and thesaurus.
[Semantic relations] What are hypernyms VS hyponyms?
Semantic relation: hyponym - denotes a subtype. Hyponymy (from Greek hupó - "under", and ónuma, "name").hypernym - denotes a supertype. Hypernymy (from Greek hupér - "over", and ónuma - "name").
[Semantic relations] What's meronyms VS holonyms?
Semantic relation: meronym - denotes a part of an item. Meronymy (from Greek méros - "part", and ónuma - "name").holonym - denotes what item is contained in. Holonymy (from Greek hólos - "whole", and ónuma - "name"). In simpler terms, a meronym is in a part-of relationship with its holonym. For example, finger is a meronym of hand which is its holonym. Similarly, engine is a meronym of car which is its holonym.
What would spoon.hyponyms() return? Spoon has many meanings, - we're looking at spoon.n.01, the first noun sense of car.
dessert_spoon, wooden_spoon, teaspoon, etc. hyponym - denotes a subtype. Hyponymy (from Greek hupó - "under", and ónuma, "name").hypernym - denotes a supertype. Hypernymy (from Greek hupér - "over", and ónuma - "name").
What's tokenization VS stemming VS tagging?
tokenization - breaking up text into words, punctuation marks (and other meaningful expressions) nltk.word_tokenize("Dr. Watson, Mr. Sherlock Holmes, said Stamford introducing us.")=> ['Dr.', 'Watson', ',', 'Mr.', 'Sherlock', 'Holmes', ',', 'said', 'Stamford', 'introducing', 'us', '.'] stemming - stemming: removing affixes from words. nltk.LancasterStemmer().stem('children') => 'childr'nltk.PorterStemmer().stem('children') => 'children' tagging - associating each word in a text with a grammatic…
What does nltk.pos_tag(text) do? text = nltk.word_tokenize('Moraya zarked the dweeble with a tolchock')nltk.pos_tag(text)
It does part of speech (POS) tagging! Parts of speech are also known as word classes or lexical categories. => [('Moraya', 'NNP'), ('zarked', 'VBD'), ('the', 'DT'), ('dweeble', 'JJ'), ('with', 'IN'), ('a', 'DT'), ('tolchock', 'NN')]
What do GANs stand for (in machine learning)?
Generative Adversarial Networks
What's the general idea behind GANs (Generative Adversarial Networks)?
It generates more data on its own when we have sparse data. Imagine a criminal is on the loose. We have one image of this ruthless terrorist, but that’s it. The data, in deep learning language, is sparse for training purposes. What GANs can do here is come up with variations, of say, different angles of his face and be trained to detect that face no matter how it changes due to age or other factors. This has far-reaching effects, especially in automated recognition through surveillance cameras.
Suppose we want to tokenize "doyouseethekittyseethedoggydoyoulikethekittylikethedoggy",and we don't yet know any vocabulary! Explain how is the score constructed in our objective function. Our task is to create a tokenization that will minimize this objective function. For some writing systems, tokenizing text is made more difficult by the fact that there is no visual representation of word boundaries. For example, in Chinese, the three-character string: 爱国人 (ai4 "love" [verb], guo3 "country",…
lexicon - total up the number of characters used by each lexical item (including a boundary marker), and derivation - each derivation, to serve as a score of the quality of the segmentation Smaller values of the score indicate a better segmentation.
[Python] What are these data structures called in Python?
Tuple ('hi', 'hello') List ['hi', 'hello'] The only difference between the two is that tuples are immutable while lists are mutable. Python allows us to omit the parentheses around tuples if there is no ambiguity.
[Python] What's the performance difference between the 1st and 2nd expressions?
2nd uses a generator expression, - the data is streamed to the calling function. If the text is very large, it could be faster. Generator expression more than a notational convenience: in many language processing situations, generator expressions will be more efficient. In the 1st expression, storage for the list object must be allocated before the value of max() is computed. If the text is very large, this could be slow. In the generator expression, the data is streamed to the calling function.
[Part-of-speech tagging] How does the Brill tagging work? The process of Brill tagging is usually explained by analogy with painting. Suppose we were painting a tree, with all its details of boughs, branches, twigs, and leaves, against a uniform sky-blue background. Instead of painting the tree first and then trying to paint blue in the gaps, it is simpler to paint the whole canvas blue, then "correct" the tree section by over-painting the blue background.
Guess the tag of every word. Apply transformation rules to fix mistakes. replace NN with VB when the previous word is TO; replace TO with IN when the next tag is NNS This technique is known as Brill tagging after its inventor. The general idea is to guess the tag of each word on the first pass, then construct a set of transformation rules to fix mistakes.
[NLTK: classification] What's this function, and what's its return value?
This function is a feature extractor. { 'last_letter': word[-1] } is a feature set.
[NLTK: classification] What do these likelihood ratio mean? These ratios are known as likelihood ratios, and can be useful for comparing different feature-outcome relationships.
This listing shows that the names in the training set ending in a - are female 38 times more often than they are male, but ending in k - are male 31 times more often than they are female.
What are 3 Pythagorean means?
Arithmetic mean Geometric mean Harmonic mean
Precision is true positives over all items we marked as positive: TP / (TP + FP).Recall is true positives over all items we should have marked as positive: TP / (TP + FN). What is F-measure?
F-measure is a harmonic mean of precision and recall. These are 3 Pythagorean means - Arithmetic Mean, Geometric Mean, and Harmonic Mean: F-measure is HM(recall, precision), and consequently has the formula:
When are confusion matrices useful?
When we want to know which types of misclassifications are frequent. E.g. the confusion matrix on the left indicates that common errors include: substitution of NN for JJ (for 1.7% of words), and substitution of NN for NNS (for 1.5% of words). Note that periods (.) indicate cells whose value is 0, and that the diagonal entries - which correspond to correct classifications - are marked with angle brackets. When performing classification tasks with three or more labels, it can be informative to s…
[Probability] State the chain rule in probability.
[Probability] What this formula translates to in probability theory?
The chain rule: P(w1:n) means P(w1 and w2 ...and... wn).
Generative VS discriminative classifiers?
These are 2 main approaches in statistical classification. We want to find P(Y|X): Generative classifier - finds P(X|Y) and P(Y). E.g. Naive BayesDiscriminative classifier - directly finds P(Y|X). E.g. Logistic Regression In statistics, classification is the problem of identifying to which of a set of categories (sub-populations) a new observation belongs, on the basis of a training set of data containing observations whose category membership is known. Examples are assigning a given email to t…
Name 3 points in the history of language models (LMs)? Language models are systems that predict the likelihood of a token (character, word or string) given its preceding or surrounding context.
LMs were proposed by Shannon in 1949, and implemented in the 1980s. n-grams used large amounts of data word-embeddings came next (word2vec, and later LSTM models such as context2vec and ELMo) Increasing the number of model parameters, however, did not yield noticeable increases for LSTMs.neural networks (transformers models such as BERT, GPT3, etc.)
What's a phrase structure grammar?
Constituency grammar In a broader sense, phrase structure grammars are also known as constituency grammars.
What's a constituency grammar VS dependency grammar? Constituency grammar == phrase structure grammar.
Dependency grammar The defining trait of phrase structure grammars is thus their adherence to the constituency relation, as opposed to the dependency relation of dependency grammars.
Lemmatization vs stemming?
Lemmatization - turns a word into its lemma (a dictionary form) am, are, is - have the lemma beStemming - turns a word into its stem (not necessarily a valid lemma) Stemming is the simplified version of lemmatisation.
How to estimate the probability of the sentence with PCFG? PCFG - Probabilistic context-free grammar