Individuals are evaluated on their knowledge or expertise in a myriad of settings. Students take exams, job candidates are interviewed on their domain knowledge, consultants help firms make decisions and are often rewarded based on the ex-post accuracy of their advice, and so on. In this dissertation I consider the problem of designing these tests “optimally”, so as to maximize the examiner’s learning of the quality of the test-taker. I take a mechanism design view of this problem, modeling knowledge as beliefs over an unknown state.

I consider three environments that differ in terms of the features of this state and the nature of knowledge. In Chapter 2, Dasgupta (2024a), I consider the most basic form of this problem where the state is binary and knowledge is the test-taker’s single-dimensional belief over it. I show that optimal tests are simple: They take the form of True-False, weighted True-False or True-False-Unsure, regardless of the principal’s preferences, the distribution of the agent’s beliefs, its correlation with his quality or his knowledge thereof. The need to elicit knowledge forces the principal to trade-off the efficacy of the test in terms of whom it rewards, against how much it rewards them. The optimal resolution of this trade-off may lead to a partial penalty for an “obvious” answer even if it is correct, a partial reward for a “counterintuitive” answer even if it is incorrect, or a reward for admitting ignorance. When the principal can pick the subject matter, she picks one that admits no such obvious answers. In this case, the highly prevalent True-False test is always optimal, regardless of principal’s preferences, agent’s learning, or the specific optimal choice of the subject matter.

In Chapter 3, Dasgupta (2024b), I consider the same problem and largely the same setting, but now knowledge is demonstrable, modeled as verifiable evidence. The test-taker learns about the state through two kinds of opposing verifiable signals, each kind providing evidence in favor of one of the states. A high quality agent is more likely to posses evidence which is greater in both quantity and accuracy, than a low quality agent. In a symmetric setting, I show that the under the optimal test, regardless of whether the agent can predict the state correctly, he is passed if his total amount of evidence provided is sufficiently high and failed if it is sufficiently low. Conditional on providing intermediate levels of evidence, the agent is passed based on a simple True-False test – i.e., if and only if he gives the correct answer. Consequently, for intermediate levels of quality sensitivity of the principal, the optimal test is the simple True-False, which makes no use of verifiable evidence, even though it is available.

In Chapter 4, Dasgupta (2024c), I consider a natural extension of the model of Chapter 2, where I allow the state to take multiple values. This captures both the cases where there are multiple questions in the test and the one where the test is about a complex subject matter instead of a simple, binary one. I show that in a symmetric setting, the standard multiple choice question – where the test-taker is awarded full credit if and only if he selects the correct answer – is optimal, if and only if the principal is sufficiently quality-sensitive.