How Does Mobi Make Great Place-Based Recommendations?

It’s the end of a long day of travel and you’re famished (maybe you skipped dinner to see a show). You’re looking for a “late night snack” close by to eat something fast and call it a night. If you’re in a city that you don’t know, chances are you’re going to search for something on your phone. The action is so mundane that we forget or never realize that search, as a computational problem, can be one of the most complex and resource-intensive processes we ask of computers. Engineers put a great deal of effort into making “search” as a problem both more computationally tractable (easier to compute) and more seamless as a user experience. Often, they do this by preprocessing data and using tailored, “domain-specific” search algorithms (where a domain could be, for example, “clothes shopping,” “flight selection,” or “song search”). Domain-informed search not only makes algorithms more efficient, but improves the quality of search results. At Mobi.AI our domain includes “travel” and “real-world experiences,” so we leverage data about timing, surroundings, and cultural context to retrieve information that feels situated in a specific time and place.
Domain-informed search is why if you are looking for a restaurant to get a “late night snack,” you will get faster and higher quality results with a dedicated app or website than by putting the query into a generic search engine. Without domain-informed search (for example if I use a generic search engine) the search cannot and doesn’t incorporate any additional context. Often, being more explicit or verbose with these systems doesn’t sufficiently improve search results. In contrast, when using a restaurant-tailored search, we can improve the results by filtering the data pool to only look at restaurants that can get food quickly (leveraging context), are currently open (leveraging time), and are a short distance away (leveraging space) before retrieving matches for the phrase “late night snack.” Mobi’s search can do this because we’ve built infrastructural layers dedicated to understanding geospatial (space) and temporal (time) information. Specifically, this “understanding” is incorporated into both the preprocessing and data retrieval stage. At Mobi, we call our search system that provides the best possible search results based on a person's particular location, time, and interests: Place-Based Recommendations.

We use geospatial and temporal data to develop a context around our search request for a physical location. People do this naturally all the time when making recommendations! When you stop someone on the street and ask them “where should I go to eat?” they, without thinking about it, build a spatial and temporal context around your question. Temporally, they are incorporating the time of day (maybe it’s lunchtime), weekday (maybe it’s Sunday, so certain things are closed), and weather (it’s a beautiful day! Maybe you want to sit outside) to narrow their response. They’re also making assumptions about your location; they’re going to give you a recommendation close by or on the way to your next destination, and using the transportation you have readily available. By adding this context, humans eliminate the absurd, the difficult, and the irrelevant. We immediately filter from millions of potential restaurant suggestions to a few dozen that can be reasoned over or compared.

Computers do not typically do this “context building” in the process of executing a basic search algorithm. There are some technical reasons for this. Understanding them is not essential to understanding the larger take-aways here, but I’m including it anyways for the curious. The best machine learning-related search methods, which use word embeddings trained on domain-specific data, incorporate only limited context from training data. These preprocessed embeddings can only incorporate approximate context into the search retrieval, struggle to incorporate explicit spatial or temporal reasoning, and can’t be fine-tuned. Different search approaches, like querying a relational database or executing a “keyword search” on a document store, allow the user to be more specific, but retrieve matches to “input” words as rigidly as possible; while results can be “filtered” on specific data qualities, these filters need to be manually constructed by the user. And all of this is, of course, still contingent on having good data that is preprocessed using a process called “data indexing.”

Mobi retrieves data using the open-source OpenSearch search engine which combines the best data-indexing, keyword search, and neural search methods. Then we add a custom “place-based context” layer that automates the construction of “filters” for a given search. By adding this layer, Mobi mimics human contextual-reasoning behavior so we can produce better tailored results.

Let’s step through an example. If I’m standing on the corner of 33rd st and 5th Ave in New York at 9PM in January and I search for “pizza,” we have a time, place, and request to start a query. If the “place-based context” layer is working as we hope it will, our search shouldn’t just return the “highest rated” pizza restaurant within five miles. I should get a recommendation for a classic New York Slice at an open restaurant that doesn’t require me to wait or eat outside in the cold within a ten minute walk. A search result that meets this criteria is good because it fits well into the time, location, and context. To get this recommendation, we have a layer that uses the initial query to construct a series of place-based filters and constraints before the OpenSearch engine then retrieves matches using the query, filters and constraints.

How do we create this filter layer? The first set of filters are the constraints that logically follow from an expansion of the request; if I am looking for pizza at 9PM in NYC, I want to filter out search results that close before 9:30, filter out results not in NYC, and filter out destinations that do not sell pizza. For the next set of filters, we pull relevant location information from geospatial and temporal data associated with pre-generated H3 cells. This type of geospatial data processing was popularized by rideshare services and allows us to quickly filter by “travel time” radius, the “vibe” of a neighborhood, weather-compatibility, and even local trends (this is the neighborhood where you can find XYZ attraction). Some of these H3 cells rely on processing libraries of open source geospatial information. Other times we use Natural Language Processing techniques to enrich our geospatial data with textual information. In this example, our location belongs to multiple H3 cells, which in turn could tell us what is within a 10 minute walk, that the vibe of this neighborhood is a bit touristy, or that there are quite a few “interesting” landmarks to walk past. These tags are then combined to generate a more specific, structured query. “Pizza at 9PM around this GPS location” expands into “New York Style Pizza, within 10 minutes walking of this GPS location, with inside seating, open until at least 9:30.” This request is then processed using the OpenSearch engine to retrieve a restaurant from pre-indexed data.

This all comes together to respond to search requests with recommendations that feel like they capture a time and place. This approach towards data retrieval is not just important for making good Place-Based Recommendations, but is also an important piece in Mobi’s larger goal of implementing Intent-Driven Search. If Mobi wants to respond well to complex user prompts like “mom and pop-owned, asian takeout with fortune cookies and online ordering on my way to my friend’s house” by properly deconstructing the prompt and addressing each component of the request, excellent place-based data retrieval is an essential capability. Our Place-Based Recommendations allow us to provide our partners with the best possible recommendations that join state of the art search technology with the insights of Human-Collaborative AI.