{ "cells": [ { "cell_type": "markdown", "id": "19adc919-63f6-4843-9991-6163a263066d", "metadata": {}, "source": [ "# 16 Non-negative Matrix Factorization" ] }, { "cell_type": "markdown", "id": "7f70625c-1f9f-4562-ac17-3681d1780691", "metadata": {}, "source": [ "Non-negative matrix Factorization is an unsupervised algorithm that simultaneously performs dimensionality reduction and clustering.\n", "\n", "It's possible to use it in conjunction with TF-IDF to model topics across documents." ] }, { "cell_type": "markdown", "id": "62e13297-8199-417f-9493-169fdc81856d", "metadata": {}, "source": [ "## Assumptions\n", "- Just like LDA, it requires a number of exptected topics (k)\n", "- Just like LDA, the result is interpreted based off the coefficient values od the words per topic.\n", "\n", "## Steps\n", "1. Construct vector spact model for documents (after stopword filtering), resulting in a term-document matrix A.\n", "2. Apply TF-IDF term weight normalisation to A.\n", "3. Normalize TF-IDF vectors to unit length.\n", "4. Initalise factors using NNDSVD on A.\n", "5. Apply projected Gradient NMF to A." ] }, { "cell_type": "markdown", "id": "5dfe5c7b-af17-4fde-883a-75dd5b05c82b", "metadata": {}, "source": [ "## Result\n", "\n", "This will result in W and H matrix.\n", "\n", "W will represent the topics belong the top frequency words and H will represent the topics belong the documents." ] }, { "cell_type": "markdown", "id": "abf81b1b-501f-492e-bd1b-2812b6a6c91f", "metadata": {}, "source": [ "## Python" ] }, { "cell_type": "code", "execution_count": 3, "id": "fbb38337-e304-4ff4-be0c-c2808fdf030d", "metadata": {}, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 4, "id": "5c562c68-aeb4-4a6d-a758-0a55189f38e2", "metadata": {}, "outputs": [], "source": [ "npr = pd.read_csv('npr.csv')" ] }, { "cell_type": "code", "execution_count": 5, "id": "720ea8c6-1181-46a7-8aac-da48d7f8f668", "metadata": {}, "outputs": [], "source": [ "from sklearn.feature_extraction.text import TfidfVectorizer" ] }, { "cell_type": "code", "execution_count": 6, "id": "84b433a9-a506-4bf1-9abf-1624b403e0ca", "metadata": {}, "outputs": [], "source": [ "tfidf = TfidfVectorizer(\n", " # max document frequency\n", " max_df=0.95,\n", " # must be present in 2 documents\n", " min_df=2,\n", " stop_words='english'\n", ")" ] }, { "cell_type": "code", "execution_count": 7, "id": "befa32e3-1acd-4c5a-ba71-9f6217cdeb9d", "metadata": {}, "outputs": [], "source": [ "dtm = tfidf.fit_transform(npr['Article'])" ] }, { "cell_type": "code", "execution_count": 8, "id": "389754b1-00e8-441b-b2e0-a80417f61a52", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "<11992x54777 sparse matrix of type ''\n", "\twith 3033388 stored elements in Compressed Sparse Row format>" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dtm" ] }, { "cell_type": "code", "execution_count": 9, "id": "7cd5456b-53fb-46cb-bb04-093a375d513d", "metadata": {}, "outputs": [], "source": [ "from sklearn.decomposition import NMF" ] }, { "cell_type": "code", "execution_count": 10, "id": "91b84ca4-4486-4725-bc81-480aada19f4a", "metadata": {}, "outputs": [], "source": [ "nmf_model = NMF(\n", " n_components=7,\n", " random_state=42\n", ")" ] }, { "cell_type": "code", "execution_count": 13, "id": "983f27ed-6de0-4fac-84fc-65fea45270f2", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

NMF(n_components=7, random_state=42)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

" ], "text/plain": [ "NMF(n_components=7, random_state=42)" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nmf_model.fit(dtm)" ] }, { "cell_type": "code", "execution_count": 14, "id": "be9309d9-62e9-484b-a545-3d1ba6d1c98c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'albala'" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tfidf.get_feature_names_out()[2300]" ] }, { "cell_type": "code", "execution_count": 17, "id": "96cdccd2-2957-46e3-83b1-f5e07b026377", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The top 15 words for topic #$0\n", "['new', 'research', 'like', 'patients', 'health', 'disease', 'percent', 'women', 'virus', 'study', 'water', 'food', 'people', 'zika', 'says']\n", "\n", "\n", "The top 15 words for topic #$1\n", "['gop', 'pence', 'presidential', 'russia', 'administration', 'election', 'republican', 'obama', 'white', 'house', 'donald', 'campaign', 'said', 'president', 'trump']\n", "\n", "\n", "The top 15 words for topic #$2\n", "['senate', 'house', 'people', 'act', 'law', 'tax', 'plan', 'republicans', 'affordable', 'obamacare', 'coverage', 'medicaid', 'insurance', 'care', 'health']\n", "\n", "\n", "The top 15 words for topic #$3\n", "['officers', 'syria', 'security', 'department', 'law', 'isis', 'russia', 'government', 'state', 'attack', 'president', 'reports', 'court', 'said', 'police']\n", "\n", "\n", "The top 15 words for topic #$4\n", "['primary', 'cruz', 'election', 'democrats', 'percent', 'party', 'delegates', 'vote', 'state', 'democratic', 'hillary', 'campaign', 'voters', 'sanders', 'clinton']\n", "\n", "\n", "The top 15 words for topic #$5\n", "['love', 've', 'don', 'album', 'way', 'time', 'song', 'life', 'really', 'know', 'people', 'think', 'just', 'music', 'like']\n", "\n", "\n", "The top 15 words for topic #$6\n", "['teacher', 'state', 'high', 'says', 'parents', 'devos', 'children', 'college', 'kids', 'teachers', 'student', 'education', 'schools', 'school', 'students']\n", "\n", "\n" ] } ], "source": [ "for index,topic in enumerate(nmf_model.components_):\n", " print(f'The top 15 words for topic #${index}')\n", " print([tfidf.get_feature_names_out()[i] for i in topic.argsort()[-15:]])\n", " print('\\n')" ] }, { "cell_type": "code", "execution_count": 18, "id": "c9447462-ae67-4849-bb50-89559313fd27", "metadata": {}, "outputs": [], "source": [ "topic_results = nmf_model.transform(dtm)" ] }, { "cell_type": "code", "execution_count": 22, "id": "19081e04-e7d8-4b18-a358-50bde9c5a557", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "topic_results[0].argmax()" ] }, { "cell_type": "code", "execution_count": 23, "id": "c2b9ac77-44a9-4fdb-8a4b-3757bc2da34a", "metadata": {}, "outputs": [], "source": [ "npr['Topic'] = topic_results.argmax(axis=1)" ] }, { "cell_type": "code", "execution_count": 24, "id": "5c1ec809-48d9-4d8b-adf4-25b2e82a5bbd", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

	Article	Topic
0	In the Washington of 2016, even when the polic...	1
1	Donald Trump has used Twitter — his prefe...	1
2	Donald Trump is unabashedly praising Russian...	1
3	Updated at 2:50 p. m. ET, Russian President Vl...	3
4	From photography, illustration and video, to d...	6

\n", "

" ], "text/plain": [ " Article Topic\n", "0 In the Washington of 2016, even when the polic... 1\n", "1 Donald Trump has used Twitter — his prefe... 1\n", "2 Donald Trump is unabashedly praising Russian... 1\n", "3 Updated at 2:50 p. m. ET, Russian President Vl... 3\n", "4 From photography, illustration and video, to d... 6" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "npr.head()" ] }, { "cell_type": "code", "execution_count": 25, "id": "5983c9b4-a302-4132-92cb-ed2037bc651f", "metadata": {}, "outputs": [], "source": [ "topics = {0:'health',1:'election_1',2:'legislation',3:'politics',4:'election_2',5:'music',6:'edu'}\n", "\n", "npr['Topic Label'] = npr['Topic'].map(topics)" ] }, { "cell_type": "code", "execution_count": 26, "id": "6844b568-aeaf-47a9-aeb6-dd5b21275205", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "

	Article	Topic	Topic Label
0	In the Washington of 2016, even when the polic...	1	election_1
1	Donald Trump has used Twitter — his prefe...	1	election_1
2	Donald Trump is unabashedly praising Russian...	1	election_1
3	Updated at 2:50 p. m. ET, Russian President Vl...	3	politics
4	From photography, illustration and video, to d...	6	edu
5	I did not want to join yoga class. I hated tho...	5	music
6	With a who has publicly supported the debunk...	0	health
7	I was standing by the airport exit, debating w...	0	health
8	If movies were trying to be more realistic, pe...	0	health
9	Eighteen years ago, on New Year’s Eve, David F...	5	music
10	For years now, some of the best, wildest, most...	5	music
11	For years now, some of the best, wildest, most...	5	music
12	The Colorado River is like a giant bank accoun...	0	health
13	For the last installment of NPR’s holiday reci...	5	music
14	Being overweight can raise your blood pressure...	0	health

\n", "

" ], "text/plain": [ " Article Topic Topic Label\n", "0 In the Washington of 2016, even when the polic... 1 election_1\n", "1 Donald Trump has used Twitter — his prefe... 1 election_1\n", "2 Donald Trump is unabashedly praising Russian... 1 election_1\n", "3 Updated at 2:50 p. m. ET, Russian President Vl... 3 politics\n", "4 From photography, illustration and video, to d... 6 edu\n", "5 I did not want to join yoga class. I hated tho... 5 music\n", "6 With a who has publicly supported the debunk... 0 health\n", "7 I was standing by the airport exit, debating w... 0 health\n", "8 If movies were trying to be more realistic, pe... 0 health\n", "9 Eighteen years ago, on New Year’s Eve, David F... 5 music\n", "10 For years now, some of the best, wildest, most... 5 music\n", "11 For years now, some of the best, wildest, most... 5 music\n", "12 The Colorado River is like a giant bank accoun... 0 health\n", "13 For the last installment of NPR’s holiday reci... 5 music\n", "14 Being overweight can raise your blood pressure... 0 health" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "npr.head(15)" ] }, { "cell_type": "code", "execution_count": null, "id": "53208de9-e623-4a0d-ac7e-78ee42255e95", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.5" } }, "nbformat": 4, "nbformat_minor": 5 }