During the 90’s the number of web sites increased which resulted in a search for how to pay for the web site hosting. Internet connections improved, visitors increased and additional hardware, software and personnel were needed for this new method of communication. Businesses experimented with multiple models and ended up using advertising dollars to pay for this growth. Advertising helped to pay for content instead of visitors which affected the the functionality and focus of the web as we know it today.
Free versus Paid Content – How to Pay for Websites While Keeping Content Free
The first sites were hosted on university, government and business servers. These early sites were viewed as additional functionality of the Internet, more of an add-on to other Internet services such as email, file transfer and basic discussion groups. With low traffic and not much content, costs were not as high to run web servers at that time. As the World Wide Web (WWW) grew, technology improved, and more people were connecting and visiting sites. Sites were added, usually starting as a college project that eventually moved out and became a business. Eventually businesses started that focused on some new use of the World Wide Web and an intent to make money instead of perform research. With more usable content, more people visited websites. Due to the change needs and immature tools, creators of those sites were scrambling to keep sites updated and also handle the increased visitors. Part of this scramble was finding funding for better hardware, software and teams of people to keep all of this technology running.
In the late 90’s, there was a lot of experimentation and a boom, the dot-com boom occurred. During this boom, investors were investing to increase the capacity and capabilities of websites. However, the business case for many web sites had not been fully vetted. There wasn’t a clear consensus on how to make money on a website, just a gut feeling that there were a lot of possibilities and the potential to get rich quickly with this new method of communication. There is a common joke on the Internet about the typical web business of the late 90’s, as follows:
1. Collect Underpants
2. ????
3. Profit!!*
*From South Park Episode 1998
Sadly, a lot of businesses in the late 90’s had less coherent business plans but if an entrepreneur said he was developing a website, investors lined up to help fund the new business. There are some businesses from that time that still survive, such as Yahoo!, Google, and Amazon, but many other new businesses eventually crashed when the money didn’t roll in as expected.
On the early days of web site, the intent was to provide content free of charge for visitors. The inventor, Tim Berns-Lee, believed that content should be free and open for everyone. He believed in this idea enough that he released all of the code required for the world wide web as free code, with no royalties, patents or other payments required to use the tools. The intent was to provide tools that allowed for the free and easy exchange of information by many people around the world. This idea partly grew from the fees people paid for cable television, music, movies and other media. People wanted an alternative to these large companies that appeared to take advantage to charge high fees. People were frustrated by the consolidation of phone companies and cable companies charging for every little interaction and the desire for free content on the Internet was a push back against these companies.
Providing content for free seemed like a good idea and it helped to grow web sites during this time. The technology was so new that there was not any existing business or institution to fight back against this idea. However, with more traffic to web sites, more money was required to maintain sites. With the precedent of not charging for content, businesses had to investigate other methods of funding their web sites. Examples are newspapers, magazines and other print media attempting to share content on the web. These businesses attempted to charge for their content but with other free content available that was labeled as news, they gave up that model for a time.
There were also small businesses, and creative people, building sites that drew large audiences but who could not raise enough money to run a robust web site server. This resulted in another funding solution which had been used in the real world, advertising, to help subsidize the costs of a website. Newspapers had classified advertising, magazines ran ads and televisions had commercials that helped to reduce their costs and make their products free or low costs. This old idea was brought into the new environment of the web.
Funding through Advertising
Advertising started as display ads, which as the name says, are ads displayed somewhere on pages of a web site.This copied the display of advertising in real world media such as magazine. The ads were displayed at the top or side of a website, and would be switched out a new visitors viewed the site. Web site owners would sign up with a web advertising company, place some special code on their web site, and display ads for visitors.
Search engines realized that they could refine this idea and sell ads that were targeted to visitors based on topics they searched for. This resulted in sites like Yahoo!, AltaVista, and eventually Google selling advertising spots and associate the advertising with search results. When searching, ads would be displayed that were connected with the search, sometimes identified as ads, other times the results were blended in with the search results.
At some point, search engines realized they could also sell display ads for web sites and allow web site owners to share in the revenue. Google created a method where web site owners could host ads on their site and receive a payment from Google for the ads that were viewed. This helped advertisers on Google increase their reach. And it helped out web site owners by having a way to fund their web site. Google also used their data from searches to refine target audiences and provide ads that were a better fit for these web site visitors.
With two or three different advertising networks, and a large audience, a website owner now had a way to pay for their content while still providing it for free to their visitors. Since money was involved, some web site owners looked for loop holes to increase ad exposure (impressions and click through rates) and increase advertising revenue. The advertising networks pushed back and eventually both sides found equitable solutions for revenue. There are still people who look for loop holes in advertising and advertising companies will continue to push back in order to keep their costs low for advertisers.
Boosting Advertising Revenue – Advertisers and Websites
The world wide web continued to grow, technology improved and advertising technology changed for web sites. Pop up ads debuted, video and audio were included and ads grew closer to television commercials in presentation. Meanwhile, the advertising networks were dealing with complaints about intrusive ads, adult ads that were shown to children, or ads that were in conflict with the intent of the website, i.e. ads for adults shown on a site meant for childrer.
These changes resulted in complaints and a loss of visitors from sites due to intrusive ads. Advertising revenue dropped, which caused web site owners to complain to the advertising agencies and demand changes. Search engines and advertising agencies started review of ads and discontinuing the use of intrusive technology, such as pop up ads, and banning advertisers or web sites that took advantage of the system. Web site owners were provided additional tools that allowed them more control over the types of ads that appeared on their sites. Advertising had become an important method of revenue for web sites, and their was interest from everyone to make ads functional.
Web site owners worked on new methods of increasing traffic in order to increase their revenue as loopholes were closed by advertising agencies. Using tools like Search Engine Optimization (SEO), they would modify the contents of their web site to improve their ranking within search engines like Google. The advertising agencies invested in improving ad effectiveness in order to increase their sales. Both advertising networks and web sites displaying ads came to a similar solution, the gathering of additional data from web site visitors.
Increasing Ad Follow Through – Data Collection and Advertising Targeting
Advertising networks and advertising agencies were learning about the world wide web and becoming knowledgeable about the data that flowed to and from visitors. For traditional ads, advertising agencies learned about customers preferences for products, the types of customers, etc, by performing surveys and polls and analyzing the data. These methods of gathering data were expensive and limited. They could also cause mistakes if data was not interpreted correctly or if not enough data had been gathered about the potential customers.
On the web, software developers naturally gathered data about visitors in support of finding errors in the software. The data was stored in log files and provided information if there were issues with the web site. The logs included information like the web pages visited on the site, amount of time spent on the site, a general location of the visitor, what type of browser and operating system they were using, etc. Web sites also used cookies, little text files, to help track a visitor’s session at a web site. A web site session involves visiting several web pages on the same site. These casual web site visitors eventually became users of the sites by the requirement of a login in order to see content and connect with other people. When a visitor logged in, the web site could associate their visit information with additional data gathered from the user’s profile for the web site.
Advertising people realized the data could be used to track visitors and users in multiple ways. This raw data could be analyzed by other software and used to generate reports that summarized the visitors and the areas that interested them most on the site. Advertising networks used the data to refine the placement of ads on appropriate sites.
Advertising networks gathered this data through the use of a tag to identify the site and associate it with data gathered from people clicking on their ads. Web site owners were willing to add tags and scripts to their site so they could receive revenue from the advertising network. They also received access to the data gathered by the advertising network. Web site owners used the data to make adjustments to their sites in order to appeal to more visitors.
The gathered data was connected and used to build profiles for zip codes, genders, socioeconomic status, which was then used to guide the development of ads for specific demographics. The data was also used to decide what websites would be most effective for the ads and the desired audience.
Here is a slightly more detailed explanation of the simple methods of tracking visitors to web sites.
Web Site Logging – Web site visits are stored in logs on the web server. The logged data is used to generate reports on visitor traffic with information such as location, time spent on the site and pages visited. The data can be used to optimize the web site performance and maximize revenue from ads, products and/or services sold on the site.
Web Site Cookies -Web site cookies are little pieces of information that a web site stores on a visitor’s computer. The information is used to store login information, shopping cart content and other information about a visit. The information is part of why you don’t have to login each time, and items in shopping carts are saved when you visit again. Some web cookies can be read by multiple sites.
Web Crawlers – Web crawlers gather up lists of web sites, and the pages associated with them and store them in databases. These databases are used for searches and analysis which generate search results or reports on popular keywords for searches. Google has built a huge database of the web and is continually crawling the web to update its data and add new sites. SEO companies also have copies of the data and use it to analyze web site competitors and effective keywords for their clients.
Advertising Tags and Scripts – Advertising tags are added to a page to identify which web site owner should receive credit when an ad is displayed on the site. An advertising script pulls in the ad from a server and displays it on the site. When an ad is viewed or clicked on, the data is sent back to the advertising agency, including the identifier for the web site. The web site owner can view reports based on visitors to their site and track which web pages are visited more. The advertising agency stores the data gathered from the visitors who clicked on the ad. The data is similar to that gathered in web site logging.
Social Media – Ideal Advertising Environment
Social media sites, where people can connect with each other, have been around for a while on the web. Some of the popular sites from the 90’s and early 2000’s were Sixdegrees, Geocities, and MySpace. They used advertising to help pay for costs, and worked to increase visitor traffic in order to increase revenue. People used them and started sharing information from their lives with other people. In some cases the sites were used to promote a business or creative venture due to the easy access to a large audience in one web site.
These sites were overtaken by technology that made it easier for anyone to post text, pictures and videos in a central location. Even with the newer technology for sites like Facebook, it took several years before they grew a large audience. In the majority of these sites, the major source of funding was advertising. Investors provided seed funding but the business plan depended on visitor traffic and advertising revenue.
As the new social media sites grew, they added the ability for visito users to advertise on their site, becoming a new type of Internet advertising network. With many visitors posting content, they were able to gather even more data that could be used to refine advertising and improve their revenue. They also had data to show potential advertisers the effectiveness of their ads and allow them to choose very specific demographics for their ads. This meant that advertisers could be more efficient and see better results for less money.
Due to the large audience on these sites, and the sharing of personal data, sites like Facebook were able to gather even more data about visitors. Social media continued adding new technology, such as Liking posts, sharing posts, and notifying other users when new content was posted. All of these activites increased the type of data they were gathering. All of the data, the posts, comments, content that was liked, shared, blocked and followed, it was all added to a database and associated with an individual user. The data was used to build a profile of each person and then used to predict what other things they might like or dislike.
Businesses were interested in access to this information for targeting their ads to a very specific audience. When a business can target a specific audience and have a higher response rate, it increases their revenue while reducing advertising costs. Before the World Wide Web, advertising would be aimed at a demographic but businesses would sometimes be guessing if they would reach that market with television, newspaper or radio ads. The percentage of people who responded to the ad was also lower so more ads were required in order to pay for an advertising campaign.
With the World Wide Web and the gathering of basic data, businesses could better refine their ad campaign and reach more people more effectively. When social media started hosting ads and building up profiles, an advertising campaign could be run with almost pin point precision in targeting potential customers. The difference was between a large blast of data sent out to many people in the hopes of a small percentage of response to an ad targeted to a specific audience. An example of the blast method of advertising is junk mail or ads on the Superbowl. Both methods are meant to reach as many people as possible with a small percentage of people buying products based on the ads they have viewed. With social media, a business could choose to target all left handed red heads who live in a rural area and only people who match those criteria would see the ad on their social media news feed. Another user, perhaps a brunette living in a large city, would see a different set of ads when viewing their feed. WIth this degree of accuracy, businesses were able to advertise and see better results from their ad campaigns.
There is a focus on social media and the data it gathers, but it is not the only place where data is pulled together to build profiles. When you shop and use the store’s club for discounts, your data is gathered and associated with your profile user login for a website. When you join a company’s rewards program and provide your email address, you also allow the company to gather data about the items you buy and which ones you use your discounts for. With the low cost of data storage, companies can store information about your purchases and keep it for long periods of time to support their data analysis.
The various pieces of data can be merged and used to build different profiles. Profiles can be for certain demographics, such as age and gender, location, and income level. A company is not dependent on just the data they have gathered. The data can be sold to other companies to use for their marketing efforts. The data from multiple companies can be combined and used to build a profile of a person that includes their email address, mailing address, phone number, frequent purchases, travel habits, music preferences and other data. The data is then sold for mailing lists, email lists, and telemarketing. A purchaser can specify that they want a list of information for all residents of zip code who are of a certain age and gender. With each additional bit of data, advertising agencies build a very exact profile of many people that is used to guide ads and help businesses build a very targeted ad campaign.
As an example, there have been stories about a store chain using their customer data to predict pregnancy. They analyzed their data and found there are 25 key products that when purchased by a woman of child bearing age are a good predictor of pregnancy. However, when they sent out coupons for baby items, many women felt spied upon about their pregnancy, especially if they had not shared their pregnancy with anyone. So the chain started sending out coupon books with other ads mixed in to convey the impression that the ads were random. They still had ads for pregnancy women and new parents but with other ads for things like lawn mowers, women felt they weren’t being singled out and would use the coupons. However, the company still had solid enough information they were able to use to predict pregnancies that was accurate enough to make women uncomfortable. With uses like this, data becomes increasingly valuable for multiple reasons.
With this focus on data gathering and building profiles, social media built on their advantages. Eventually their focus on revenue generation became the selling of ads based on profile data. With this shift, it became important to keep visitors on a social media site for as long as possible. The longer a person views a site, the more ads they view and the higher likelihood that they will click on the ad and take action. Below are a few of the more commonly known methods
Never Ending News Feeds – In the early days of web sites, the content on a web site page was limited. If a page had too much text and/or pictures, it could crash the web browser. It would also slow down the computer. Technology improved and more content could be displayed without causing problems on the visitors computer. However, pages still had an end and a visitor normally had to click a link to continue to another page.
When a visitor had to take an action, they might also leave the site, especially if the next page seemed slow to load. Social media sites wanted to keep people on the site so they experimented with methods of doing this. Eventually the concept of a never ending news feed was implemented on one of the sites. The other social media sites quickly copied this idea.
A never ending news feed keeps pulling in new content and displaying it on the same web page as a visitor scrolls down the page. Since there is no break, visitors are less likely to click away from the site. The feed also contained items that had been identified as popular with the specific visitor. The intent was to continually feed in engaging content that keeps a visitor on the site.
The Algorithm – A popular term, the Algorithm is the mysterious force that identifies what content each visitor sees when they visit a social media site. Based on my understanding, the Algorithm is a software program that reviews all of the posts on a social media site and compares them with the profile of a visitor. Based on what a visitor to the site likes, who they are friends with, what groups they participate in, and many other variables, the algorithm selects posts to display in a news feed. The selection is based on finding items that will be the most engaging for a visitor and keep them on the site longer.
The Algorithm will also leave out posts that may encourage a person to leave the site. Posts without links are less likely to be displayed because people are more likely to click the link and leave the site. Posts from friends and not businesses are pushed more so that people stay involved on the page and don’t feel inundated with ‘advertising’. However, if a business pays for advertising, then the posts are boosted up and are more likely to appear in people’s feeds. Posts that invoke an emotional response are also pushed because people are more engaged with the post. If people are commenting on a post, liking it or otherwise reacting to it, they are staying on the site. The emotion itself isn’t as important, so long as people don’t go to another site. This means that posts that make people happy, joyful, angry, hateful or fearful are all viewed in the same way, they keep people engaged. In some cases, emotions like anger are actual considered a good thing because many people are likely to engage as they argue with each other in comments.
Boosting Paid for Content – At first most accounts on social media sites and their posts were treated the same. As a site grew, posts from businesses were penalized unless they paid to boost the post to their followers. Small businesses who had developed a large following would find the follow through on their posts dropped while they were encouraged to pay for boosting their posts. Personal posts were boosted equally so that there would be more content to keep visitors on the site.
’Organic Posts’ – Social media sites created a new type of ad, an organic ad. These types of posts are displayed in the same font and method that regular posts are displayed but they are paid for and boosted to a selected demographic. The posts are written so that they appear to have been written by another user of the site. These ads were created after ad blocking software was developed and used by more visitors to the sites.
Automated Posting – Tools were developed that allowed scheduling posts to appear at optimal times for viewing. This allowed content creators to plan out posts and post them ahead of time. For small businesses, these tools allowed them to appear to be on-line 24/7 and keep their content in front of multiple people. For larger businesses, they could build an entire campaign of automated posts and post them at intervals designed to maximize viewing and click through rates.
Sock Puppet Accounts – A sock puppet account is an account that is created to manipulate opinion about another account. This type of account is used to either support or attack another account and appear to be separate from the other account. The same person may be running multiple accounts and responding to posts in order to create the appearance of engagement which will boost a post’s visibility on the social media news feeds. Using sock puppet accounts can give an appearance of greater appeal for an account which can also boost up their relevance on the site.
Bot Networks – A bot network for social media consists of multiple accounts that are all controlled by one person or company. A bot network can be bought and used to like a post and cause to get boosted in popularity. This is related to the organic posts, the idea of making it appear that content is popular when in reality it is getting an artificial boost from somewhere else. By creating the impression that something is popular, the social media site would increase the likelihood that the item would show up in more persons feeds. As more real people viewed the content, it would continue to increase the impressions and eventually build on itself. In this way, fake popularity is used to boost something until it is really popular. The person who has the popular or viral item can then get revenue from advertisement on their website or possibly n the social media site. Or the increased traffic is used to increase awareness about their product or idea that is promoted. Bot networks are used more on sites like Twitter and Reddit, not quite so much on sites like Facebook.
Many of these techniques are used to promote content and help businesses. I know of several small businesses that have built up an audience and increased sales of their product because of their posts and advertising on places like Facebook, Twitter and other sites. The social media sites are fine with these techniques because they help to keep users on the site longer which helps to boost their advertising revenue. However,the use of advertising to fund web sites has had some unintended consequences that are causing problems in the real world . I will explore some of these in the next article, Impacts of Advertising on the World Wide Web.
Links and Definitions
On-line advertising with history- https://en.wikipedia.org/wiki/Online_advertising
Web Crawler/Search bot – https://en.wikipedia.org/wiki/Search_bot
Internet Bot – https://en.wikipedia.org/wiki/Internet_bot
News Feed – A news feed on social media is the personalized view displayed for each visitor to the page. The news feed is filled with posts from people or groups that the visitor has indicated they liked. With extended permissions, the news feed may also include posts from people or groups who are similar to things the user has liked in the past. Because of the data and algorithm, each visitor will have a unique view of posts when compared to others. There may be overlap, but the order and position will vary for each visitor.
Ad impressions – Advertisers pay for ads based on how many times they are displayed to an audience, I.e. how many impressions do they make for a market
Click Through Rate (CTR) – A method of calculating ad effectiveness by how many people click the ad to go to a web site for information and/or to purchase product. May also be used to charge advertisers for their ads based on how many people click on it on a web site.
Search Engine Optimization – This led to things like Search Engine Optimization (SEO), the process of adding content to a website that will improve its place in search results. By improving a websites location in results, this can lead to higher traffic to the site. SEO at its most basic level works to write content that will optimize its place in the search results. It’s much better to have a website show up in the first ten results from a search engine instead of the first 25 or 50 results. The difference between a number one position in search results and number ten could mean the differences of hundreds or thousands of visitors and dollars for the website.
In the early days, in order to improve search engine ranking, tricks were used to place keywords on a website. Techniques included putting in text that was colored white so it wouldn’t show up to visitors but would be read by a web crawler. The text might not have anything to do with the site, but it would result in the site displayed in top search results at Google. Google and other search engines eventually closed these loopholes, so SEO companies continue to look for new methods to promote sites for certain keywords.