Noun Compound Interpretation

Transcription

1 Noun Compound Interpretation by Girishkumar Ponkiya Supervisor: Prof. Pushpak Bhattacharyya Co-supervisor: Mr. Girish K Palshikar (TRDDC, Pune) June 21, 2015

2 Outlines Introduction Problem Definition Two Approaches Automatic Rule Based Deep Learning An interesting problem Noun Compound Processing 2

3 Introduction Noun Compound (NC): sequence of two or more nouns that act as a single noun. Example: apple pie, student protest Task: interpret the meaning of English NCs (bi-gram) Labeling: relationship of modifier with the head noun. Apple pie : Made-Of Paraphrasing Apple pie : a pie made of apple or a pie with apple flavor Noun Compound Processing 3

4 Motivational Example Honey Singh became the latest victim of celebrity death hoax. Machine Translation: [Hindi] हन स घ प रस द ध व यक त क म त क ब र म अफव ह क त ज स क र बन Hanī siṅgha prasid'dha vyakti kī mauta kē bārē mēṁ aphavāha kē tājā śikāra banē. Question Answering: What type of rumor was spread about Honey Singh? Text Entailment: H: Honey Singh is dead. (False) Noun Compound Processing 4

5 More Examples.. Japan May exports fall on earthquake disruption The Economic Times (June 20, 2016) Noun order in a compound: Mosquito malaria v/s malaria mosquito Adult male rat v/s male adult rat Groping is important in long sequences: "plastic water bottle" v/s "water bottle cap" Noun Compound Processing 5

6 Noun Compound Processing 1. Identification Identification of noun compounds from a sentence (or text) Student protest is a common thing in West Bengal. Kindly throw used plastic water bottles in the trash. The term weblog was first coined in 1997, 3 years after the first dynamic website was created. Some kelp products are snake oil, but the good ones promote plant growth. 2. Parsing (if necessary) tumor suppressor protein [ tumor suppressor ] protein 3. Interpretation We are going to discuss this in this presentation Noun Compound Processing 6

7 Problem Definition Given: a two-word English noun compound Output: assign an abstract label (relationship of modifier with the head) Dataset: 1. Kim and Baldwin (2005) KB05 (20 relations; 2084 compounds) Best result: 53.38% accuracy (using WordNet similarity) 2. Tratz and Hovy (2010) TH10 (43/37 relations; compounds) Best result: 79.3% accuracy (multiclass SVM), and 77.7 average F-score (using neural network) Noun Compound Processing 7

8 Semantic Relations used by Kim and Baldwin (2005) Noun Compound Processing 8

9 Challenges The compounding process is highly productive In BNC, 60.3% of total noun compounds appears only once (Baldwin and Tanaka, 2004) The semantic relation is implicit The relation of a modifier noun with the head noun in a compound is not mentioned explicitly. Contextual pragmatic factors influence the interpretation Why students are beneficiary in student price? Noun Compound Processing 9

10 Noun Compound Interpretation Illustrative Literature Survey Attributional Methods (Based on the similarity between the respective components of compounds) Relational Methods (Model relation directly) Rule Based System Deep Learning Probabilistic Modeling P(r n 1, n 2 ) or f(n 1, p, n 2 ):tri-gram using predicates Paraphrasing Using some patterns, or verbs and predicates Explicit Paraphrasing From predefines patterns, fine the best for given NC Implicit Paraphrasing Create a vector of frequency of each paraphrase, and use an algorithm to classify. Noun Compound Processing 10

11 Our Approaches Rule Based Extracted rules using CN2 Probabilistic inferencing using MLN (Markov Logic Network) Deep Learning Learn embedding for noun compounds A new problem Coherence in noun compounds Noun Compound Processing 11

12 Example: student protest Hypernym paths for each of the components entity.n.01, physical_entity.n.01, causal_agent.n.01, person.n.01, enrollee.n.01, student.n.01. entity.n.01, abstraction.n.06, psychological_feature.n.01, event.n.01, act.n.02, speech_act.n.01, objection.n.02, protest.n.01 Noun Compound Processing 12

13 Example: student protest Hypernym paths for each of the components, with default value for the rest 1. entity.n physical_entity.n causal_agent.n person.n enrollee.n student.n X 8. X 9. x 1. entity.n abstraction.n psychological_feature.n event.n act.n speech_act.n objection.n protest.n X Noun Compound Processing 13

14 CN2: inputs Attribute file: description and domain of each attribute Example file: Vector for each example Params: Error function: laplacian, naïve Star size: for bean search Threshold: stop condition Noun Compound Processing 14

15 CN2: Outputs Rule file: ordered/unordered rules An example rule: IF "u.5" = N_ AND "v.6" = N_ THEN labels = agent [ ] N_ : polity.n.02 N_ : work.n.01 Evaluation matrix Noun Compound Processing 15

16 CN2 Results Noun Compound Processing 16

17 MLN (Alchemy) Training Input: MLN definition file Domain(s) of variables (optional) predicate declaration logic in terms of formula Database file Evidence and outcomes Output: MLN file with weight assigned to each of the formulas Testing Input: MLN file with weight assigned to each of the formulas Database file: evidence only Output: Probability for each ground atom of outcome predicate Noun Compound Processing 17

18 MLN file format // Domain for each of the attributes (optional) u0 = {} u1 = {}... labels = {AGENT, BENEFICIARY, CAUSE,..., TOPIC} // Class Outcome(nc, labels!) // Predictors U0Value(nc, u0) U1Value(nc, u16)... // Rules V6Value(x, "creation.n.01") ^ U2Value(x, "group.n.01") => Outcome(x, AGENT)... Noun Compound Processing 18

19 MLN Results Noun Compound Processing 19

20 Our Approaches Rule Based Extracted rules using CN2 Probabilistic inferencing using MLN (Markov Logic Network) Deep Learning Learn embedding for noun compounds A new problem Coherence in noun compounds Noun Compound Processing 20

21 Word/Text Representation 1-hot representation Bag-of-words representation Don t capture similarity between synonyms Two words with different surface forms have orthogonal representation, i.e., dot product is 0. Dimension of a vector equals to number of words in vocabulary, which is in terms of thousands in most!! Can we have better presentation which can capture similarity between words? Noun Compound Processing 21

22 Word Embeddings Suppose you have a dictionary of words The i th word in the dictionary is represented by an embedding: w i R d i.e., d-dimensional vector which is learnt!! d typically in range of 50 to 1000 Similar words should have similar embeddings (share latent features) Noun Compound Processing 22

23 Embedding of 115 Country Names (Bordes et al., 11) Noun Compound Processing 23

24 Deep Neural Network (DNN) Deep Learning has been used for learning high-dimensional continuous-valued vector for words Idea is: can we learn similar embedding for the semantic relation in noun compounds? Fine tuning of generic word embedding constructed independently has improved the performance (Collobert et al., 2011) Fine tuning: task specific improvement of generic word embedding Noun Compound Processing 24

25 DNN: Architecture (Dima and Hinrichs, 15) Noun Compound Processing 25

26 Results (TH10) Average Precision: 0.78 Average Recall: 0.78 Average F-score: 0.78 Noun Compound Processing 26

27 Nearest Neighbors of a compound: 1/2 robot arm (WHOLE+PART_OR_MEMBER_OF) Using Component Similarity robot spider foot arm service arm mouse skull machine operator elephant leg car body Using Compound Similarity dinosaur wing airplane wing mouse skull jet engine airplane instrument pant leg fighter wing Noun Compound Processing 27

28 Nearest Neighbors of a compound: 2/2 hillside home (LOCATION) Using Component Similarity waterfront home brick home trailer home winter home boyhood home retirement home summer home Using Compound Similarity waterfront home patio furniture fairway bunker beach house basement apartment ocean water beach resort Noun Compound Processing 28

29 Analysis (zero f-score classes) Noun Compound Processing 29

30 Representation at Hidden Layers First Hidden Layer Second Hidden Layer Output Layer Noun Compound Processing 30

31 Our Approaches Rule Based Extracted rules using CN2 Probabilistic inferencing using MLN (Markov Logic Network) Deep Learning Learn embedding for noun compounds A new problem Coherence in noun compounds Noun Compound Processing 31

32 The current lingering question Any sequence of nouns cannot be a noun compound For example: Student protest Towel juice Towel namkeen So, question is: what is that thing that decides such coherence in noun compound? Noun Compound Processing 32

33 References 1/2 Timothy Baldwin and Takaaki Tanaka. Translation by machine of complex nominals: Getting it right. In Proceedings of the Workshop on Multiword Expressions: Integrating Processing, pages Association for Computational Linguistics, Renu Balyan and Niladri Chatterjee. Translating noun compounds using semantic relations. Computer Speech & Language, Ken Barker and Stan Szpakowicz. Semi-automatic recognition of noun modier relationships. In Proceedings of the 17th international conference on Computational linguistics-volume 1, pages Association for Computational Linguistics, Cristina Butnariu and Tony Veale. A concept-centered approach to noun-compound interpretation. In Proceedings of the 22nd International Conference on Computational Linguistics-Volume 1, pages Association for Computational Linguistics, Peter Clark and Robin Boswell. Rule induction with cn2: Some recent improvements. In Machine learning EWSL-91, pages , Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493{2537, Corina Dima and Erhard Hinrichs. Automatic noun compound interpretation using deep neural networks and word embeddings. IWCS 2015, page 173, Noun Compound Processing 33

34 References 2/2 Pamela Downing. On the creation and use of English compound nouns. Language, pages , Su Nam Kim and Timothy Baldwin. Automatic interpretation of noun compounds using WordNet similarity. In Natural Language Processing IJCNLP 2005, pages Springer, Preslav Nakov. On the interpretation of noun compounds: Syntax, semantics, and entailment. Natural Language Engineering, 19(03): , Parag Singla. Markov Logic: Theory, Algorithms and Applications. PhD thesis, University of Washington Graduate School, Stephen Tratz. Semantically-enriched parsing for natural language understanding. University of Southern California, Stephen Tratz and Eduard Hovy. A taxonomy, dataset, and classier for automatic noun compound interpretation. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages Association for Computational Linguistics, Noun Compound Processing 34

35 Thank you Contact: Noun Compound Processing 35