Small-molecule therapeutics treat a wide variety of diseases, but their effectiveness is often diminished because of their pharmacokinetics — what the body does to a drug. After administration, the body… Read More »Big data dreams for tiny technologies
Materials called perovskites are widely heralded as a likely replacement for silicon as the material of choice for solar cells, but their greatest drawback is their tendency to degrade relatively… Read More »Homing in on longer-lasting perovskite solar cells
The potential of artificial intelligence to bring equity in health care has spurred significant research efforts. Racial, gender, and socioeconomic disparities have traditionally afflicted health care systems in ways that are… Read More »3 Questions: Artificial intelligence for health care equity
The potential of artificial intelligence to bring equity in health care has spurred significant research efforts. Racial, gender, and socioeconomic disparities have traditionally afflicted health care systems in ways that are… Read More »3 Questions: Artificial intelligence for health care equity
Transformers have been successfully applied to a wide variety of modalities: natural language, vision, protein modeling, music, robotics, and more. A common trend with using large models is to train a transformer on a large amount of training data, and then finetune it on a downstream task. This enables the models to utilize generalizable high-level embeddings trained on a large dataset to avoid overfitting to a small task-relevant dataset.
We investigate a new setting where instead of transferring the high-level embeddings, we instead transfer the intermediate computation modules – instead of pretraining on a large image dataset and finetuning on a small image dataset, we might instead pretrain on a large language dataset and finetune on a small image dataset. Unlike conventional ideas that suggest the attention mechanism is specific to the training modality, we find that the self-attention layers can generalize to other modalities without finetuning.
Read More »Pretrained Transformers as Universal Computation Engines
Explaining, interpreting, and understanding the human mind presents a unique set of challenges. Doing the same for the behaviors of machines, meanwhile, is a whole other story. As artificial intelligence… Read More »More transparency and understanding into machine behaviors
Explaining, interpreting, and understanding the human mind presents a unique set of challenges. Doing the same for the behaviors of machines, meanwhile, is a whole other story. As artificial intelligence… Read More »More transparency and understanding into machine behaviors
In his book “Intelligence and Surprise Attack: Failure and Success from Pearl Harbor to 9/11 and Beyond,” 21-year naval intelligence veteran Dr. Erik Dahl highlights the importance of the intelligence… Read More »The Fight Against Intelligence Failures with NLP: Rosette Identity Resolver
There are some tasks that traditional robots — the rigid and metallic kind — simply aren’t cut out for. Soft-bodied robots, on the other hand, may be able to interact… Read More »Researchers’ algorithm designs soft robots that sense