حسن شامحمدی

 

دانشکده فنی و مهندسی

گروه آموزشی مهندسی کامپیوتر

اطلاعیه دفاع از  پایان نامه کارشناسی ارشد در رشته مهندسی کامپیوتر گرایش هوش مصنوعی

عنوان:

تشخیص نقل­به­مضمون با استفاده از تکنیک­های یادگیری عمیق

استاد راهنما:

دکتر میرحسین دزفولیان

  استاد مشاور:

دکتر محرم منصوری زاده

اساتید ممتحن:

دکتر حسن ختن­لو

 دکتر مهدی عباسی

پژوهشگر:

حسن شاه­محمدی

 

زمان:

یکشنبه 30/10/1397 ساعت 14:00

مکان:

سمینار 2 دپارتمان برق (سالن مهندس مرحوم خانمحمدی)

 

Bu-Ali Sina University

Faculty of Engineering

Department of Computer Engineering

 

Thesis submitted for Master of Science in computer Engineering-Artificial Intelligence

 

Title:

A novel method for point cloud analysis based on deep learning techniques 

Supervisor:

Dr. Mir Hossein Dezfoulian

 

Advisor:

Dr. Muharram Mansoorizadeh 

Judges:

Dr. Hassan Khotanlou

Dr. Mahdi Abbasi

 

Author:

Hassan Shahmohammadi

 

 January 20, 2019

 

تشخیص نقل­­به­مضمون با استفاده از تکنیک­های یادگیری عمیق

تشخیص نقل­به­مضمون یکی از مسائل مهم در حوزه پردازش زبان­های طبیعی است. نقل­به­مضمون به جملات یا عباراتی اشاره می­کند که معنی و مفهوم یکسانی را به خواننده منتقل می­کنند اما ساختار و کلمات آن­ها با هم متفاوت است. این مسئله کاربردهای فراوانی در حوزه پردازش زبان­های طبیعی دارد.  ازجمله این کاربردها می­توان به استفاده آن در خلاصه­سازی متن، ترجمه ماشینی، سیستم­های پرسش­و­پاسخ، تشخیص سرقت ادبی و موتور­های جستجو اشاره کرد. در این پژوهش، ابتدا مسئله با چندین روش مرسوم مانند وزن­دهی TF-IDF و استفاده از طبقه­بند­هایی همچون ماشین بردار پشتیبان، حل و ارزیابی شده است. سپس با استفاده از  نتایج بدست آمده از این روش­ها، یک مدل جدید برای تشخیص نقل­به­مضمون ارائه شده است. مدل پیشنهادی را می­توان به دو بخش تقسیم نمود. در بخش اول که مسئله با تکنیک­های یادگیری عمیق حل می­شود، جملات پس از عبور از مرحله پیش­پردازش، با استفاده از تکنیک پنهان­سازی GloVe به بردار­هایی عددی تبدیل می­شوند. خروجی این لایه پنهان­سازی سپس به یک شبکه Bi-LSTM برای بیان کردن کل جمله داده می­شود. پس از اتمام آموزش مدل، خروجی این شبکه به‌عنوان ویژگی­های استخراج شده برای هر جمله در نظر گرفته می­شوند. در بخش دوم، یک سری ویژگی دستی برای بیان کردن میزان شباهت معنایی بین دو جمله معرفی می­شوند. از میان این ویژگی­ها، تعدادی از آن­ها جدید بوده و برای اولین بار در این پژوهش معرفی شده­اند. مدل پیشنهادی از ترکیب ویژگی­های بدست آمده در این دو بخش حاصل می­شود. دو مجموعه داده با نام­های MSRP و Quora برای ارزیابی مدل پیشنهادی در نظر گرفته شده­اند. نتایج مدل برای مجموعه داده MSRP نشان می­دهد که این مدل تقریبا از تمام پژوهش­های انجام شده، کارایی بهتری از نظر صحت و f-measure را کسب می­کند. نتایج ارزیابی مدل برای مجموعه داده Quora نیز کارایی قابل قبول و قابل مقایسه­ای با سایر پژوهش­های انجام شده روی این مجموعه داده را نشان می­دهد. به طوری که مدل پیشنهادی جزء 24 درصد برتر روش­ها از میان بیش از 3000 تیم در سایت Kaggle است. نتایج ارزیابی همچنین نشان می­دهد که مدل پیشنهادی برای مجموعه داده­هایی با تعداد نمونه­های کم، کارایی بهتری در مقایسه با سایر مدل­های جدید دارد.

 

Paraphrase detection using deep learning techniques

Paraphrase detection is one of the fundamental tasks in the area of natural language processing. Paraphrase refers to those sentences or phrases that convey the same meaning but use different wording. It has a lot of applications. For instance, it could be used in machine translation, text summarization, QA systems, plagiarism detection, and search engines. In this research, we fist employed several common classification and feature extraction methods to deal with the problem. Models used in this step are simple, for example TF-IDF term weighting and SVM classifier are employed.  In the second step, advantages and disadvantages of each simple model is scrutinized. Then, a new model is introduced based on the conclusion of the results of the simple models. The new model can be divided into two parts. In the first one, each sentence pass through a preprocessing step and then its constitutive words convert to their numerical representation using GloVe word embedding. The output of the GloVe word embedding is then fed into a Bi-LSTM neural network to encode the whole sentence by leveraging its word vectors. In the second part, three sets of handcrafted features are used to measure the similarity between each pair of sentences. Some of these features are new and introduced in this research for the first time. The new model is formed by combining the handcrafted features and the output of Bi-LSTM network. Two datasets are used to evaluate the model namely, MSRP and Quora. The result of evaluating the model on MSRP shows that it outperforms almost all the previous works in terms of f-measure and accuracy. The evaluation results on Quora also shows comparable results among other works. On Quora-question pair competition launched by Kaggle, our model ranked 24% top solutions among more than 3000 teams. The results of evaluation for both datasets shows that our model achieves the best relative results when the training data is not huge. In other words, it can generalize well despite the lack of adequate training data for deep models.

 

 

Paraphrase detection using deep learning techniques

Paraphrase detection is one of the fundamental tasks in the area of natural language processing. Paraphrase refers to those sentences or phrases that convey the same meaning but use different wording. It has a lot of applications. For instance, it could be used in machine translation, text summarization, QA systems, plagiarism detection, and search engines. In this research, we fist employed several common classification and feature extraction methods to deal with the problem. Models used in this step are simple, for example TF-IDF term weighting and SVM classifier are employed.  In the second step, advantages and disadvantages of each simple model is scrutinized. Then, a new model is introduced based on the conclusion of the results of the simple models. The new model can be divided into two parts. In the first one, each sentence pass through a preprocessing step and then its constitutive words convert to their numerical representation using GloVe word embedding. The output of the GloVe word embedding is then fed into a Bi-LSTM neural network to encode the whole sentence by leveraging its word vectors. In the second part, three sets of handcrafted features are used to measure the similarity between each pair of sentences. Some of these features are new and introduced in this research for the first time. The new model is formed by combining the handcrafted features and the output of Bi-LSTM network. Two datasets are used to evaluate the model namely, MSRP and Quora. The result of evaluating the model on MSRP shows that it outperforms almost all the previous works in terms of f-measure and accuracy. The evaluation results on Quora also shows comparable results among other works. On Quora-question pair competition launched by Kaggle, our model ranked 24% top solutions among more than 3000 teams. The results of evaluation for both datasets shows that our model achieves the best relative results when the training data is not huge. In other words, it can generalize well despite the lack of adequate training data for deep models.

 

 

 

Hassan Shahmohammadi

 

 

·         BSc	Software engineering , University of Kurdistan,  September 2012 – August 2016 Thesis title: An android application to full control personal computers via Wi-Fi connection.
·         MSc	Artificial intelligence, University of Bu-Ali Sina, September 2016 – December 2018 (expected) Thesis title: Paraphrase detection using deep learning techniques.

___________________________________________________________________________________

 

Research Interests 

 

·         NLP: QA and recommender systems, short text analysis, semantic text representations, word embedding, sentiment analysis, topic modeling, recognizing textual entailment, paraphrase identification.

 

·         ML: Deep Learning models, ensemble techniques.

 

·         DIP: Image tagging, image captioning, image classification.

 

·         2008, Attending in Final Level of 4th Iran National Astronomy and Astrophysics Olympiad

 

·         Working at Parsecoders.com (an Iranian version of freelancer.com) since August 2017 as a programmer.

___________________________________________________________________________________________

Sample Projects

·         Artist identification using baseline CNNs and transfer learning (using inception v-3).

·         Sentiment analysis using LSTM networks on IMDB dataset.

·         Paraphrase detection using varies feature extraction methods such as word2vec, GloVe, pharagraph2vec, TFIDF, etc.

·         Short text similarity using convolutional neural networks.

·         Generating text using LSTM networks.

·         Estimating age from photo using AlexNet.

·         Representation of short texts using unsupervised methods.

·         Kaggle competition: https://www.kaggle.com/c/dogs-vs-cats

·         Kaggle competition: https://www.kaggle.com/c/quora-question-pairs

Github: https://github.com/Hazel1994

_____________________________________________________________________________________

Teacher Assistant

·         Theory of Formal Languages & Automata(undergraduate)  

·         Algorithm design (undergraduate)

·         Machine learning( graduate)

 _______________________________________________________________________________________

 

Courses Taken

 

·         Statistical Pattern Recognition, Ref: Introduction to Statistical Pattern Recognition, By Keinosuke Fukunaga

 

·         Advanced Artificial Intelligence, Ref:  Learning from data, by Yaser S. Abu-Mostafa

 

·         Natural language processing, Ref: Speech and Language Processing (2nd Edition), by Daniel Jurafsky and James H. Martin

 

·         Machine learning, Ref: Introduction to machine learning (third edition), by Tom Mitchell

 

·         Evolutionary Computing, Ref: An introduction to genetic algorithms, by Melanie Mitchell

 

·         Expert systems, Ref: Expert systems principles and programming (third edition), by Joseph C. Giarratano

 

·         Information retrieval, Ref: an introduction to information retrieval, by manning raghavan and Schutze

 

·         Digital Image Processing, Ref: Digital Image Processing (3rd Edition), by Rafael C. Gonzalez and Richard E. Woods

________________________________________________________________________________________________

 

 

 

 

 

 

 

 

References

 

Hassan Khotanlou, Associate Professor

Department of Computer Engineering, Bu-Ali Sina University of Hamadan.

PhD from Pierre & Marie Curie University, Paris, France.

Email: hkh@basu.ac.ir,

Homepage: http://profs.basu.ac.ir/khotanlou/index.php?L=en&pc

 

MirHossein Dezfoulian, Assistant Professor

Department of Computer Engineering, Bu-Ali Sina University of Hamadan.

PhD from University of Wollongong, Australia.

Email: dezfoulian@basu.ac.ir,

Homepage: http://profs.basu.ac.ir/h-dezfoulian/index.php?L=en&pc

 

Muharram Mansoorizadeh, Assistant Professor

Department of Computer Engineering, Bu-Ali Sina University of Hamadan.

PhD from Tarbiat Modares University, Tehran, Iran.

Email: mansoorm@basu.ac.ir,

Homepage: http://profs.basu.ac.ir/mansourizadeh/index.php?L=en&pc