Comparison of BQ Answers

Status
Not open for further replies.

Portugal

Newcomer
Mar 1, 2014
17
1
3
Think about it, if you wanted to determine the post hiring effectiveness of the BQ, then you would give it to ALL applicants but not use it as a elimination tool. You would then follow all the individuals through training etc. and retroactively measure the washout rate based upon their previous BQ results. Common sense?
 

GS3k

Trusted Member
Oct 24, 2013
410
16
18
Think about it, if you wanted to determine the post hiring effectiveness of the BQ, then you would give it to ALL applicants but not use it as a elimination tool. You would then follow all the individuals through training etc. and retroactively measure the washout rate based upon their previous BQ results. Common sense?
Sure thing, just issue your theory to a peer reviewed journal, have it empirically confirmed and then have a leg to stand on.

"Only white males in the '90s exhibit certain personality traits" by Portugal.
 

Portugal

Newcomer
Mar 1, 2014
17
1
3
*Clears throat* "Who cares if they're all white obviously they're the most qualified."
That's the point. They are eliminating people -good people who are probably the MOST qualified because they don't fit into this norm that they've established. Military individuals may answer the questions differently than non-military. Women will answer the questions different than men...same with different cultures. If you were raised in a Hispanic or Asian household and spoke the native language at home then it's common sense that your high school grades in English average lower than others. The point is that many of the questions on the BQ shouldn't have been used because they're biased. If the norm was established by polling 87% minority female ATC's in the 90's, then more than likely a white male applicant today would fall outside of the norm band.

I am not trying to bring up race, culture or gender here as a complaint. I'm only saying that the BQ is a joke and should have never been used. Take the BQ to a local agency that performs testing for police or fire departments and ask them to review the questions...I did. They'll laugh you out of the office because so many of the questions are not neutral.

When you automatically dismiss so many of the ex military people with VERY high qualifications, then something is wrong.
 

Portugal

Newcomer
Mar 1, 2014
17
1
3
Sure thing, just issue your theory to a peer reviewed journal, have it empirically confirmed and then have a leg to stand on.

"Only white males in the '90s exhibit certain personality traits" by Portugal.
Are you asserting that females and those of different culture or race don't have different personality and biographical traits than white males? Is it not established that men perform better in math while women perform better in English. Whites have higher high school scores than blacks and Hispanics? Blacks don't have a higher unemployment rate than whites? Hispanics and Asians who live in a first generation household and speak their native language at home don't have lower scores in English? Military individuals have greater assurance and dominance than non military? Women are less dominant than men?

The BQ had several questions that covered all these areas. You have to remember that they're trying to match the "average" response of a universe of individuals. And when you're washing out 13 out of 14 applicants based upon their deviation from this "average," the differences in gender, culture (military), race can have a pronounced effect. It's construct was terrible.
 

GS3k

Trusted Member
Oct 24, 2013
410
16
18
Are you asserting that females and those of different culture or race don't have different personality and biographical traits than white males? Is it not established that men perform better in math while women perform better in English. Whites have higher high school scores than blacks and Hispanics? Blacks don't have a higher unemployment rate than whites? Hispanics and Asians who live in a first generation household and speak their native language at home don't have lower scores in English? Military individuals have greater assurance and dominance than non military? Women are less dominant than men?

The BQ had several questions that covered all these areas. You have to remember that they're trying to match the "average" response of a universe of individuals. And when you're washing out 13 out of 14 applicants based upon their deviation from this "average," the differences in gender, culture (military), race can have a pronounced effect. It's construct was terrible.
You have to be very careful when you try to take broad generalizations and apply them to the individual. Take me for example, English was not my first language and yet I managed to score a 680/800 on the English portion of the P.S.A.T. in the 7th grade. I also tested out of two semesters of college English in high school through the advanced placement program. My female cousin is an aeronautical engineer for Boeing and her sister / my other cousin is following suit by getting her Masters in Engineering as well. Not everyone in the military is the Type A personality, and not every woman is submissive.

I think the point of what I'm trying to tell you is that it would take a statistician and/or psychologist with years of schooling and experience to try to disprove a test of this nature that has years of research and peer review behind it. I think attempting to do so is Quixotic at best.
 

CL600AMG

Junior Member
Oct 14, 2010
114
1
18
Answered and Submitted. This BQ was almost the same as the one we had take on the AT-SAT. honestly if we already passed the AT-SAT and the BQ that went with it, why the .... do we have to do it again. And how in the world can they say that this time we are sorry you failed it.... what changed. I will be really excited to see what the results will be when you compare it.

Funny thing I had my friend apply for the announcement just for shits and giggles and he completely trolled on the BQ and with the resume. Congratz he passed and is moving on to the next round. AGE 29 3 different associates in medical field and secretary work in medical field and they selected him. Now he is actually thinking about accepting and going on to the next phase. I will post screen shots as soon as I can for proof.
 

Portugal

Newcomer
Mar 1, 2014
17
1
3
You have to be very careful when you try to take broad generalizations and apply them to the individual. Take me for example, English was not my first language and yet I managed to score a 680/800 on the English portion of the P.S.A.T. in the 7th grade. I also tested out of two semesters of college English in high school through the advanced placement program. My female cousin is an aeronautical engineer for Boeing and her sister / my other cousin is following suit by getting her Masters in Engineering as well. Not everyone in the military is the Type A personality, and not every woman is submissive.

I think the point of what I'm trying to tell you is that it would take a statistician and/or psychologist with years of schooling and experience to try to disprove a test of this nature that has years of research and peer review behind it. I think attempting to do so is Quixotic at best.
Your using an individual example - yourself -to try to nullify proven averages. Let's examine just one of the BQ questions... How many months were you unemployed during the last 36 months? I think that we can all agree, and hundreds of studies will confirm..including government statistics, that blacks in the targeted age group have a CONSIDERABLY higher unemployment rate than whites. So statistically, a black applicant answering that question will be outside of the norm of the 87% white universe that was polled to derive the norm....it's not rocket science. The fact is that there were several questions on the BQ that were not gender, racial or culturally neutral. And when you're only taking 13 out of 14 applicants to move further in the process, that small deviation in answers can have a substantial effect.
 

Portugal

Newcomer
Mar 1, 2014
17
1
3
Your using an individual example - yourself -to try to nullify proven averages. Let's examine just one of the BQ questions... How many months were you unemployed during the last 36 months? I think that we can all agree, and hundreds of studies will confirm..including government statistics, that blacks in the targeted age group have a CONSIDERABLY higher unemployment rate than whites. So statistically, a black applicant answering that question will be outside of the norm of the 87% white universe that was polled to derive the norm....it's not rocket science. The fact is that there were several questions on the BQ that were not gender, racial or culturally neutral. And when you're only taking 13 out of 14 applicants to move further in the process, that small deviation in answers can have a substantial effect.
And regarding years of research and peer review behind it.... Could you please cite your source for that statement. Where is the years of research and peer review behind the 60 question BQ? Nobody has been able to find any review of this assessment other than the July 2012 review done by the FAA itself that states that it's validity still needs to be determined.
 

Portugal

Newcomer
Mar 1, 2014
17
1
3
My error on the previous post where I said that they're only passing 13 out of 14 applicants...I meant DQing 13 out of 14. I should have said that when they're only passing 1 out of 14 individuals that take the BQ, the small deviations due to non neutrality of questions can have a pronounced effect on the outcome...ESPECIALLY on an instrument that's only 60 questions. Most personality inventories are longer to reduce this effect.
 

GS3k

Trusted Member
Oct 24, 2013
410
16
18
It's funny that you use that particular question regarding unemployment to detract from the B.Q. because I was unemployed for eight, almost nine months before the job posting went up and I answered the question honestly when it came up. I still passed. You're grasping at straws.
 

GS3k

Trusted Member
Oct 24, 2013
410
16
18
As far as studies in regards to the B.Q. I'm not talking about this B.Q. in specific but using these types of assessments to gauge the success of employees in general. The only way we will know whether or not the B.Q. was successful will be when the F.A.A. releases data on the 2014 O.T.S. hires years from now.

I've also heard that the Academy is supposedly changing their approach and will be more stringent in the future.
 

Portugal

Newcomer
Mar 1, 2014
17
1
3
It's funny that you use that particular question regarding unemployment to detract from the B.Q. because I was unemployed for eight, almost nine months before the job posting went up and I answered the question honestly when it came up. I still passed. You're grasping at straws.
GS3k you just don't get it...I think the subject may be a little to difficult to comprehend. We're talking about averages here and there's equal bias to several questions. You keep bringing up your individual circumstance.

Let's try it this way... If they gave the BQ to several groups of 100 controllers each that were hired during the last five years and were categorized by race, gender and military vs non military, do you honestly believe that the average answers for several of the questions would be the same for each group? They would not. There would be a disparity in answers for several of the questions. A group of 100 female controllers would have a different average answer to several of the questions compared to a group of 100 male controllers. Same with a black, vs Hispanic vs white. This is what the FAA was SUPPOSED TO DO to ensure the validity of the test. Then they would discard the questions that have the greatest gender or race bias and construct an instrument that is neutral.

Imagine if they had originally obtained their baseline data from a group of controllers in the 90's comprised of 87% Hispanic. Then let's assume for simplicity's sake that a white takes the present BQ and one of the questions used to determine if he/she fits the "profile" of the controllers that were polled is 'how proficient he or she is in a second language.' Since the "average" controller (87% Hispanic) is 100% proficient in two or more languages, then, accordingly, that must be a good predictor of success as a controller. If the white answered anything except 100%, then he/she WOULD NOT fit the profile of a successful controller. While I don't know the statistics, I'm presuming that MOST whites aren't 100% proficient in two or more languages. This example is exaggerated but I'm trying to explain the premise that the questions asked on the BQ weren't neutral...they were based on a universe of white males.

So unless the BQ questions are racially, culturally, gender neutral, the results will be skewed and, on average, those applicants answering the questions most consistent with that of a white male will be chosen to proceed. These disparities are especially pronounced when the assessment is only 60 questions long and they take the 10% that are most consistent with the norm.

For the record, I'm a white male.
 

AK02

Senior Member
Apr 8, 2012
157
9
18
GS3k you just don't get it...I think the subject may be a little to difficult to comprehend. We're talking about averages here and there's equal bias to several questions. You keep bringing up your individual circumstance.

Let's try it this way... If they gave the BQ to several groups of 100 controllers each that were hired during the last five years and were categorized by race, gender and military vs non military, do you honestly believe that the average answers for several of the questions would be the same for each group? They would not. There would be a disparity in answers for several of the questions. A group of 100 female controllers would have a different average answer to several of the questions compared to a group of 100 male controllers. Same with a black, vs Hispanic vs white. This is what the FAA was SUPPOSED TO DO to ensure the validity of the test. Then they would discard the questions that have the greatest gender or race bias and construct an instrument that is neutral.

Imagine if they had originally obtained their baseline data from a group of controllers in the 90's comprised of 87% Hispanic. Then let's assume for simplicity's sake that a white takes the present BQ and one of the questions used to determine if he/she fits the "profile" of the controllers that were polled is 'how proficient he or she is in a second language.' Since the "average" controller (87% Hispanic) is 100% proficient in two or more languages, then, accordingly, that must be a good predictor of success as a controller. If the white answered anything except 100%, then he/she WOULD NOT fit the profile of a successful controller. While I don't know the statistics, I'm presuming that MOST whites aren't 100% proficient in two or more languages. This example is exaggerated but I'm trying to explain the premise that the questions asked on the BQ weren't neutral...they were based on a universe of white males.

So unless the BQ questions are racially, culturally, gender neutral, the results will be skewed and, on average, those applicants answering the questions most consistent with that of a white male will be chosen to proceed. These disparities are especially pronounced when the assessment is only 60 questions long and they take the 10% that are most consistent with the norm.

For the record, I'm a white male.
You, along with all the rest of us, have no idea how the test was developed. So we are all arguing a moot point.
 

Portugal

Newcomer
Mar 1, 2014
17
1
3
You, along with all the rest of us, have no idea how the test was developed. So we are all arguing a moot point.
That's incorrect. You can find the link on a previous post to a FAA analytical that describes how it was developed. What is most interesting is the conclusion stating that more research needs to be done to determine its validity.
 

AK02

Senior Member
Apr 8, 2012
157
9
18
That's incorrect. You can find the link on a previous post to a FAA analytical that describes how it was developed. What is most interesting is the conclusion stating that more research needs to be done to determine its validity.
I posted that thread and read the whole thing. It never said that it was the developing factor for the BQ for this announcement.
 

e4rls

Newcomer
Feb 14, 2014
5
0
1
Here is another interesting thought, does anyone have any proof (other than their say so) that they actually ONLY used the bio test to determine result?
It appears that they pretty consistly picked young people with little or no ATC experience - I am not sure the bio test could have done that consistantly. To be clear I have no evidence, and am not saying that everyone picked has no experience; I am just speculating.
 

AK02

Senior Member
Apr 8, 2012
157
9
18
Here is another interesting thought, does anyone have any proof (other than their say so) that they actually ONLY used the bio test to determine result?
It appears that they pretty consistly picked young people with little or no ATC experience - I am not sure the bio test could have done that consistantly. To be clear I have no evidence, and am not saying that everyone picked has no experience; I am just speculating.
i would refer you to http://www.stuckmic.com/faa-atc-february-open-announcement-bid/25576-any-veteran-preference-applicant-make.html plenty of experienced controllers have moved on.
 

psykomagician

Junior Member
Jan 10, 2014
114
8
18
And I'll bet you FAA has got a study demonstrating the Biographical Assessment has (some degree) of predictive validity. FAA isn't stupid on these issues, folks. They've been sued before (and won and lost). They've been going round and round with Dr. Jim Outtz (the 1st author of the Barrier Analysis) who is also a Fellow of the Society for Industrial Organizational Psychology (SIOP, the primary professional organization for personnel psychology), multiple publications, etc. etc. So I'll bet they've done their homework on validity and fairness for the Biographical Assessment.
 

psykomagician

Junior Member
Jan 10, 2014
114
8
18
Portugal - How do you know its the same instrument? Have you, for example, contacted the authors (one of whom is an FAA employee) to ask if they are the same? Seems a simple thing to look someone up on Google who has publications like that.
 

gump

Newcomer
Jun 22, 2008
23
0
1
Hey Fellows,

This is the Gumpster. Here's the REAL deal. Ready?

If you only hired Air Traffic Controllers...You would have ONLY Air Traffic Controllers...BAM
 
Status
Not open for further replies.