CareerCast JobSerf Employment Index: Methodology

The CareerCast.com/JobSerf Employment Index has tracked the online job market since January 2007. At the start of each month, the research team searches for jobs on select major job boards and also on job search engines (sometimes referred to as aggregators). Using search words to retrieve jobs for different levels of management across dozens of geographies (both cities and states), the teams then review the returns to determine the number of real, relevant jobs that exist in each geography, at each career level, and on each job site.

Using a mathematical formula to accommodate for level importance, geography and job site contribution, the total is summarized in an index format, the CareerCast.com/JobSerf Index, which will be released monthly. The CareerCast.com/JobSerf Index is a comparison of online job postings in a current month to the same month in 2007, so as to eliminate the distorting impact of cyclicality of the job market throughout the year.

What does it tell me?

The CareerCast.com/JobSerf Index is a barometer showing the change in management level positions across the internet as compared to the same month going back to 2007. Using the Index, one can see if things are getting worse (current month versus previous month), compare year to year (present month to same month in previous year), see trends and forecast economic performance based on employment data.

How We Calculate Our Index

In an effort to create a single reflective value of the change in volume of jobs posted online, the CareerCast.com/JobSerf Employment Index combines data from a mix of both major job boards and job search engines. Each month our research team conducts searches for specific geographies using a standard set of keywords, and records the number of total listings returned for each keyword. Then our teams analyze the search results to determine the number of "real" jobs on each site. Using a weighted system to normalize the results, a single composite value for the U.S. Index is calculated.

BASE YEAR

BASE COMPARISON YEAR

We selected 2007 as our base year because it was the last year of relatively stable nationwide unemployment figures, with only a 0.5 % variation throughout the 12 calendar months (Bureau of Labor Statistics). Current job posting volume is measured against base year figures on a 100 point scale, with 100 equal to the posting volume of a particular month from 2007.

DATA POINT DEFINITIONS

DATA DEFINITIONS & VARIABLES

There are four defining variables for each data set we collect:

• Month & Year
• Website (Job Board or Job Search Engine)
• Geography (City/Metro Area or State)
• Career Level

Month/Year

For each month of the year, we gather our entire range of data across all surveyed websites, geographies and levels.

Website (Job Board or Job Search engine)

We gather data from a variety of the major job boards and job search engines in our Index, utilizing the U.S. version of each site.

Geography (City/Metro Area or State)

We use a comprehensive set of geographies, including the largest cities/metro areas as well as a number of states. Over 75% of the total U.S. population is represented by our geographical selections.

Career Level

For each geographical area (City or State) on a specific board, we assess four different career levels: C-Level (CEO, CFO, etc.), Vice Presidents (all levels), Directors (all levels, except Managing Director which falls under C-Level) and Manager. To avoid duplication, we only use one keyword or string for each data set.

Within each data set, we capture or calculate three data elements:

• a) Total job listings returned per keyword (i.e. Career Level
• b) Total number of ‘real' or relevant jobs
• c) Percentage of ‘real' jobs versus total listings returned

a) Total job listings returned per keyword (All)

Upon entering the appropriate keyword and the desired geography, each job site provides the number of the listings of jobs it currently contains in its database. We capture this number, and refer to it as ‘All', for the total number of listings.

b) Total number of ‘real' or relevant jobs

The total number of listings per keyword is then analyzed to determine the number of ‘real' or relevant jobs. These include any jobs which are actual positions for the corresponding level. Work at home opportunities, retail career assistance or other advertisements in the database are excluded, as well as any listings which do not fall under our job level specifications. For example, "Assistant to the CFO" or "Sales experience to CEOs" would be eliminated from our C-Level statistics.

c) Percentage of ‘real' jobs versus total listings returned

We capture the percentage of real jobs and divide them by the total number of job listings returned. For larger returns (some can exceed 1000), our teams use a statistical sample (the first 300 listings returned), and then multiply it back over the total number. For example, if a search on Manager in California returns 2,651 job listings, the first 300 would be analyzed to determine the number of "real" or relevant jobs. If the number is 234, it would equate to 2,068 "real" jobs (234/300 x 2,651).

BOARDS

JOB BOARDS & JOB SEARCH ENGINES

The Index research team evaluated dozens of major job boards and job search engines for inclusion in the Index. After careful analysis, a representative subset reflecting over one million online jobs was selected. The Index separates major job boards from search engines, given that in raw numbers the search engines' data would overwhelm the major boards. The Index is also similar to the Dow Jones Industrial Average, in that its structure allows for components to be both added and removed reflecting the current environment. Job search engines are weighted higher since they often represent thousands of boards and have a higher percentage of "real" jobs.

The following is the weighting of the job board and job search engine groups.

BOARD WEIGHTING SYSTEM

With the overall split between the two groups made above, individual allocation for Boards and Engines are determined factoring in size and current marketplace conditions.

GEOGRAPHIES

GEOGRAPHICAL SELECTION & REPRESENTATION

The geographical selections use a state and major metropolitan area combination. Some of the reasons for this include concern for accuracy when major metropolitan areas span multiple states (eg New York, D.C., Chicago). In addition, the relevancy of more specific data to the user (reader) was felt to be limited at the state level, and of greater interest for metropolitan areas. Also, the team originally designed the index using the Federal Reserve Districts as a base, and selected approximately 40 cities and states whose inclusion could be made ‘cleanly' within each of the twelve districts, representing more than 230 million people. Given the high labor requirement of our data gathering technique, some smaller states were not included given the low chance of any material change to the Index.

The following is a breakdown of the geographical areas which are included, representing more than 230 million people across the U.S.

LEVELS & KEYWORDS

LEVEL ANALYSIS & KEYWORD SELECTION

The following are the approximate weightings of the four different managerial levels measured by the Index.

KEYWORD EXECUTION

For each job board and job search engine, our teams have a single search word or string of words, which may vary from board to board, but our data gathering is consistent. The objective is to obtain the greatest number of job listings returned without conducting multiple searches so as to avoid duplication.

TIMING

TIMING OF INDEX DATA GATHERING

Data collection is initiated on the first weekday of each month, with teams assigned to specific job boards to ensure consistency.

COUNTING

When counting, our teams look for the same positions listed twice within a board, and then only include them once. For the job search engines, we do not include jobs from those boards already counted from our Major Job Board groups, in order to avoid double counting.

CALCULATION

To calculate the U.S. Index, we first add up all of the data points for a specific board and level combination across all included geographies. For example, the values for all states and cities for the Director level of Job Board A would be added together for a single value.

Then each one of these values (there are four for each of the Boards and Job Search Engines we include) is multiplied by the weight allocation for that particular Job Board, and then it is multiplied by the weight assigned to its particular Level.

The final U.S. Index is the sum of all of these values. Both the weights for the Boards and the Levels are on 100 point scales, so this allows us to readily create a 100 point base value using the data.

More Questions? Check out our Index FAQ File