Overview
Recommendation Systems are automated systems that suggest relevant users or items to users based on similarities in their behaviors. In this article, I will walk you through working and implementing a User to User Collaborative Filtering Recommendation System using Python. So let’s get started.
What is User-Based Collaborative Filtering
User-based Collaborative Filtering is a memory-based widely used technique in Recommendation systems to recommend people of similar interests to each other. For example, on any social media app, a user may display his interests to be sports and movies, and there may be other users having precisely the same interests i.e. sports and movies, so those users are more likely to be recommended to the first user based on their similar taste and interest.
How User-Based Collaborative Filtering Works
Now let’s understand how User to User Collaborative Filtering Algorithm works.
As you can see,
User 1 likes Marvel and DC
User 2 likes Starwars
User 3 likes Marvel and Starwars
So the recommendation would work like this,
User 1: User 3
User 2: User 3
User 3: User 1 and User 2
Step by Step implementation
Now let’s begin python implementation of the above-discussed approach.
Data Preprocessing
Firstly, explore the dataset.
Now apply NLP preprocessing techniques like lowercasing, removing stop words, lemmatization, etc to preprocess user interests.
| import nltk from nltk.corpus import stopwords from nltk.stem import WordNetLemmatizer import re def text_lower(data): return data.lower() def stopword(data): nltk.download(‘stopwords’) clean = [] for i in data: if i not in stopwords.words(‘english’): clean.append(i) return clean def lemmatization(data): nltk.download(‘wordnet’) lemma = WordNetLemmatizer() lemmas = [] for i in data: lem = lemma.lemmatize(i, pos=’v’) lemmas.append(lem) return lemmas def remove_characters(data) data = re.sub(r'[^a-zA-Z0-9, ]’, ‘ ‘, data) |
and the results would look like this
Word Embeddings
In this step, we vectorize our preprocessed text for the next step.
| from gensim.models import KeyedVectors import numpy as np model_path=”/files/glove-wiki-gigaword-50.gz” model_wiki = KeyedVectors.load_word2vec_format(model_path) #choose from multiple models https://github.com/RaRe-Technologies/gensim-data def get_vector(data): return np.sum(np.array([model_wiki[data]]), axis=0) |
Its results would look like this.
Clustering
Now, we will insert user embeddings to our clustering algorithm and it will return clusters
| from sklearn.cluster import AgglomerativeClustering aglo = AgglomerativeClustering(n_clusters=None, affinity=’cosine’, linkage=’complete’,distance_threshold=0.8) agg_cluster=aglo.fit_predict(result_array) print(agg_cluster) |
Cosine Similarity
Cosine Similarity can be used as an alternative to clustering when computing distance between users to specify when ones are closer or similar to each other based on their features. Its results can be seen as a similarity matrix or CSR matrice. It helps better in setting priorities.
| from sklearn.metrics import pairwise_distances def get_cosine_similarity_score_with_prev_users(df): try: # print(df) # ”’Computing Cosine Similarity of User Vector with Prevoius Users”’ values_array=df.values # print(values_array) dist_out = 1-pairwise_distances(values_array, metric=”cosine”) similarity_with_user = pd.DataFrame(dist_out,index=df.index) similarity_with_user.columns=df.index similarity_with_user=similarity_with_user.round(decimals = 4) return True,similarity_with_user except Exception as e: print(e) return False,0 |
Result
Using one of the above two approaches, you will be able to implement a recommendation system from scratch.
Final Words
It’s one of the valuable automation techniques readily used in every other high-traffic app be it online stores, media apps, social media, music apps, or games. Recommendation systems are being used everywhere, improving the app’s traffic and user experience.
In AI based recommendation systems, it is used for personalized recommendations of what to watch next, in social media apps, it tells who might be your next best friend, in e-commerce stores, it suggests what will suit you better, in the music app, it will pick your next favorite song for you, and so on. There are countless use cases of where and how recommendation systems can be integrated into a business to elevate it.
So, in this article, you have learned why you use a recommendation and how easy it is to implement and integrate it with your existing app only using the app data.
