Participants in both the four-level "good" table and the four-level "reliable" table conditions were significantly more likely to report the message from the color-coded reliability indicator than the three-level indicator in a post-hoc test. Participants did use the reliability indicator and frequently said that they would report the message contained in the reliability column along with the estimate (12% indicator reporting for the three-level condition and 43% reporting for each of the four-level conditions). Eye-tracking heat maps showed that participants looked at the MOE column on the table fewer times for the three-level condition. An analysis of how likely participants were to mention the margin of error (MOE) or report it along with the estimate showed no significant differences by table design. There is evidence that satisfaction as measured by the QUIS instrument was significantly higher for both the baseline and the four-level tables than for the three-level table. However, there were some significant dfferences in satisfaction scores between the tables. Overall, 19 of the 21 participants indicated that they preferred the prototype tables over the baseline tables, although their preference for either the three- (n=10) or four-level (n=9) tables or either of the two alternative wording options was fairly evenly split (nine preferred the "good" wording, ten preferred the "reliable" wording, and two had no preference). One version of the four-level table was labeled with the terms "excellent," "good," "fair," and "poor." The other was labeled with the terms "reliable," "mostly reliable," "somewhat reliable," and "unreliable." No differences were found in users' accuracy and effciency (time on task) across the four tested table designs. One prototype had three levels of reliability and the other two had four levels of reliability. Three prototype tables with a color-coded indicator based on an estimate's coefficient of variation were compared to a baseline table that represented a past production version of an ACS data profile. This study was the second round of usability testing for the Data Reliability Indicator for American Community Survey (ACS) data tables proposed by the sponsor team.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |