Semantic technologies are at the heart of the future Web providing ways to express knowledge and data so that it can be properly exploited. These technologies will empower a new class of Information and Communication Technologies much more scalable, interoperable, and with a higher degree of process automation support that will fulfill the needs of an emergence market that will exceed $10 billion by 2010. This is a very active research area, but still suffers from a lack of standard benchmarks and infrastructures for assessing research outcomes. SEALS addresses two key challenges: the creation of a lasting reference infrastructure for semantic technology evaluation and the continuous benchmarking of semantic technologies at a large scale via public worldwide evaluation campaigns.