DLTM

Summary

Real estate listing descriptions offer insight into market inventory and perceived buyer preferences. Overall themes in the listings and the evolution of themes through time are an important summary of local market conditions. Real estate markets exhibit complicated dynamic behavior, and learning time-varying themes from listings is challenging for two reasons: first, the listings are textual descriptions of a home; and second, the descriptions reflect the local market at a single point in time. In this brief, we describe the Dynamic Linear Topic Model, a new method of learning from text that allows us to overcome both of these challenges. In the listings data, we learn themes related to location, outdoor spaces, renovations, views, and more. For each theme, we learn a unique time-varying pattern. Interactive visualizations of the learned topics illustrate their dynamic behavior.

Unstructured data is a rich source of information

Data can typically be categorized as either structured or unstructured. Structured data, like that found in spreadsheets and relational databases, has a standard format. The prices of homes in a neighborhood are an example of structured data: they can be presented in a two column table with one column for an identifier of each house and the second column for the price. As a result, structured data is easy to query, sort, and explore with standard tools. By contrast, unstructured data -- like pictures, video, and text -- lacks a standard format for presenting information. Idiosyncracies in form and style mean that identifying valuable pieces of information is more difficult. The challenge of learning from unstructured data is pushing the frontiers of artificial intelligence, machine learning, and statistics.

At Zillow, structured data has been the driver of innovations such as the Zestimate and Zillow Home Value Index (ZHVI). Directly quantifiable characteristics of a home like square footage, year built, and number of bedrooms facilitate analytical comparisons of one property to another, which is useful for buyers, sellers, and real estate professionals; however, the unique character that distinguishes a house from a home is difficult to quantify with numbers alone. The feeling of home is more often evoked by pictures, videos, and text descriptions.

In this post, we describe one way that important features of a home can be extracted from listing descriptions. Unstructured listing descriptions complement pictures and videos by offering information and context that visual media cannot fully provide. As an example, details about renovations and original elements of the home can be easily conveyed with text, whereas critical updates to plumbing, heating/cooling, and electrical systems are not easily captured by pictures. The same is true of specific details about building materials such as hardwood flooring, cabinets, and countertops. For some buyers, oak floors and granite counters are attractive features, but the actual building materials are not always clear in pictures. Important aspects of the neighborhood and surrounding environment are also included in text descriptions. Terms like family-friendly, vibrant, and diverse are keywords that shed light on the nature of a community.

Real estate listings exhibit complicated time-varying patterns

The goal of this analysis is threefold: first, we want to learn the important themes in a particular market from a set of real estate listings and understand how these themes change over time; second, we want to understand the relative importance of each theme in the overall market; and third, we want to understand how much each theme contributes to individual listing descriptions.

One way of learning themes from text is with topic models. Topic models represent themes in text with probabilities; a topic is defined as a probability distribution over all the possible words that could be used to create a document. A real estate topic about kitchens may place high probability on words like appliances, stainless steel, and granite but very low probability on words like garden, patio, and backyard. On the other hand, a topic focused on outdoor spaces may place high probability on garden, patio, backyard, but give very little weight to kitchen-related terms. Our first goal is to learn these topics from the listing descriptions.

One particularly important element of real estate markets is that they are always changing. Evolution in market conditions is reflected in prices, the amount of time needed to sell a home, and even in the listing description. Sellers and agents describe a home to make it most appealing in a dynamic marketplace. Emphasizing a home's garden when selling in the summer or it's fireplace when selling in winter is a natural way of appealing to current buyer preferences. While some market themes may recur seasonally, others may significantly increase (decrease) in importance from one year to the next, like when neighborhoods become more desirable or fall out of favor. Such market trends can persist for many years. Our second goal is to learn the complicated dynamic patterns that topics exhibit.

While some Zillow users are interested in overall market patterns, most are looking for a specific home. The choice of a home is very personal: some prefer homes that are cozy and cute; others are looking for renovated classics. Ideally, the unique features of a home match the individual preferences of the buyer. Classifying individual listings by their themes may lead to improved recommendations to Zillow users. This is why our third goal is to understand how much each topic contributes to individual listings.

Real estate listings from Zillow

The data used for this analysis is a collection of more than 11,000 listing descriptions from Seattle, Washington from November of 2007 to March of 2017. Each listing is summarized by the number of times each word appears. This bag of words approach is standard in topic modeling, and it facilitates the dynamic modeling strategy discussed below.

Dynamic Linear Topic Models (DLTM)

To describe the statistical method used to learn the topics and their dynamic patterns, let's think about a procedure for writing a new listing . The procedure begins with a vocabulary, which is the predefined set of words that can appear in all the listings. Next, we choose the number of themes that we want the collection to contain. For simplicity, let's imagine a set of listings containing two different themes: location and indoor spaces. Each of these topics will correspond to a different bag of words. We can think of each bag containing ping pong balls marked by vocabulary words. For example, in the bag of words associated with location, there will be many ping pong balls with words like neighborhood and school so that if we reach into the bag and randomly select a ball, it is likely that the word on the selected ball will describe the location.

Let's write a new listing in March 2018 with 100 words. To choose the first word in the listing, we need to reach into one of the two bags to draw a ping pong ball. Which one do we choose? In this model, we choose the bag by flipping a coin. If we flip heads, we reach into the location bag. If we flip tails, we choose from the indoor spaces bag. Let's say we flipped tails, and we choose the first word in our listing by reaching into the indoor spaces bag and randomly drawing one of the ping pong balls. It's marked kitchen. Kitchen is the first word in our new listing.

To write the second word, we repeat the process by flipping the same coin again to choose one of the two bags. Let's say this time we flipped heads, and we draw the word downtown from the location bag. Downtown is our second word. This process repeats until the hundredth word is selected and the new listing is complete.

In June 2018, we want to write another new listing. To represent the passage of time, the bags of words are slightly changed. In the location bag, there are a few more ping pong balls with the term lake and a few less ping pong balls with highway. The indoor spaces bag will also slightly change. In addition to the bags themselves changing, the coin we flip to choose the bag also changes. Our new coin may not have the same probability of flipping heads as the original coin. For example, instead of a 50-50 chance of flipping heads with the original coin, suppose that there is a 70 percent chance of flipping heads with our new coin. With both the bags of words and the coin changed by the passage of three months, we again write the document by flipping the coin, choosing a bag, and selecting a ping pong ball from the bag.

In this example, we are writing new listings with a randomized procedure. When analyzing the actual data, we reverse the process. The listings are observed, but we do not observe the bags of words or the coin used to choose the bags. We must learn them. Our first goal is to use the listings data to estimate the percentage of ping pong balls in the location bag associated with each word. We estimate the percentages in each bag of words at each time point. This corresponds to learning the topics and how the topics change over time. Our second goal is to learn the probability of flipping a heads (i.e. selecting the location topic). We want to learn the probability of each topic at each time point. In models with more than two topics, the process is similar to flipping a coin with many sides. This corresponds to learning the importance of each topic in the corpus as a whole and how that importance changes over time.

The innovation of the Dynamic Linear Topic Model is the ability to model complicated patterns in the probability of selecting each topic. Imagine that we want to use the same coin to select topics in March 2019 as we do in March 2018. It is possible to model this repeated use of the same coin as seasonality in the topic probabilities. It is also possible to model probabilities that rapidly increase or decrease from one month to the next with polynomial trends.