Ir al contenido principal

Understanding Artificial Intelligence through Algorithmic Information Theory

Can we characterize intelligent behavior?
Are there theoretical foundations on which Artificial Intelligence can be grounded?

This course on Algorithmic Information will offer you such a theoretical framework.

  • You will be able to see machine learning, reasoning, mathematics, and even human intelligence as abstract computations aiming at compressing information.
  • This new power of yours will not only help you understand what AI does (or can’t do!) but also serve as a guide to design AI systems.
Understanding Artificial Intelligence through Algorithmic Information Theory
Este curso está archivado
Se anunciarán próximas fechas
5 semanas estimadas
4–8 horas por semana
A tu ritmo
Avanza a tu ritmo
Verificación opcional disponible

Sobre este curso

Omitir Sobre este curso
  • Artificial Intelligence is more than just a collection of brilliant, innovative methods to solve problems.
    If you are interested in machine learning or are planning to explore it, the course will make you see artificial learning in an entirely new way. You will know how to formulate optimal hypotheses for a learning task. And you will be able to analyze learning techniques such as clustering or neural networks as just ways of compressing information.
  • If you are interested in reasoning , you will understand that reasoning by analogy, reasoning by induction, explaining, proving, etc. are all alike; they all amount to providing more compact descriptions of situations.
  • If you are interested in mathematics , you will be amazed at the fact that crucial notions such as probability and randomness can be redefined in terms of algorithmic information. You will also understand that there are theoretical limits to what artificial intelligence can do.
  • If you are interested in human intelligence , you will find some intriguing results in this course. Thanks to algorithmic information, notions such as unexpectedness, interest and, to a certain extent, aesthetics, can be formally defined and computed, and this may change your views on what artificial intelligence can achieve in the future.

Half a century ago, three mathematicians made the same discovery independently. They understood that the concept of information belonged to computer science; that computer science could say what information means. Algorithmic Information Theory was born.

Algorithmic Information is what is left when all redundancy has been removed. This makes sense, as redundant content cannot add any useful information. Removing redundancy to extract meaningful information is something computer scientists are good at doing.

Algorithmic information is a great conceptual tool. It describes what artificial intelligence actually does , and what it should do to make optimal choices. It also says what artificial intelligence can’t do. Algorithmic information is an essential component in the theoretical foundations of AI.


Algorithmic information, Kolmogorov complexity, theoretical computer science, universal Turing machine, coding, compression, semantic distance, Zipf’s law, probability theory, algorithmic probability, computability, incomputability, random sequences, incompleteness theorem, machine learning, Occam's razor, minimum description length, induction, cognitive science, relevance.

De un vistazo

  • Institución: IMTx
  • Tema: Informática
  • Nivel: Advanced
  • Prerrequisitos:

    Examples of what you should know before embarking on this course

    • what a convex curve looks like,
    • that log(7^n) is n times log(7)
    • that rational numbers have finite or periodic expansion,
    • that rational numbers are countable, but that real numbers are not,
    • that the probability of "A and B" is the probability of "A knowing B" times the probability of B,
    • that 65 is 1000001 is in base 2 and 41 in base 16,
    • how to compute the sum of a finite geometric series,
    • that {'a':1, 'i':0} is a Python dictionary and why list('ab'*4)[::2] yields ['a','a','a','a'],
    • that k-means is a clustering method,
    • what Bayes’ theorem tells us,
    • how Shannon’s information is related to probability,
    • that what is called a Turing machine is NOT the machine that Alan Turing (Benedict Cumberbatch)
      is using in the movie The imitation game.
  • Idioma: English
  • Transcripción de video: English

Lo que aprenderás

Omitir Lo que aprenderás
  • How to measure information through compression

  • How to compare algorithmic information with Shannon’s information

  • How to detect languages through joint compression

  • How to use the Web to compute meaning similarity

  • How probability and randomness can be defined in purely algorithmic terms

  • How algorithmic information sets limits to the power of AI (Gödel’s theorem)

  • A criterion to make optimal hypotheses in learning tasks

  • A method to solve analogies and detect anomalies

  • A new understanding of machine learning as a way to achieve compression

  • Why unexpected means abnormally simple

  • Why coincidences are unexpected

  • Why subjective information and interest are due to complexity drop and why relevance, aesthetics, emotional intensity and humour rely on coding.

Caveat: This course DOES NOT address the notion of "computational complexity" which measures the speed of algorithms.

Plan de estudios

Omitir Plan de estudios

Chapter 1. Describing data

  • Complexity as code length
  • Conditional Complexity

Chapter 2. Measuring Information

  • Complexity and frequency
  • Meaning distance

Chapter 3. Algorithmic information & mathematics

  • Algorithmic probability, Randomness
  • Gödel’s theorem

Chapter 4. Machine Learning and Algorithmic Information

  • Universal induction - MDL
  • Analogy & Machine Learning as complexity minimization

Chapter 5. Subjective information

  • Simplicity & coincidences
  • Subjective probability
  • Relevance

Testimonios de los estudiantes

Omitir Testimonios de los estudiantes

"The cognitive part was really exciting..." (Antoine, 2021, betatester)

"Yes, the overall course was very interesting!" (Joachim, 2021, betatester)

"A very interesting course that definitely changed how I see the world. Thank you so muchhh." (Hanady, 2021, betatester)

Acerca de los instructores

Preguntas frecuentes

Omitir Preguntas frecuentes

Do I need to be highly proficient in Python?

No. Basic knowledge of Python is sufficient. From time to time, you’ll have to use short Python programs, to understand them locally and to perform easy transformations.

I am feeling a bit rusty in maths. Is it a problem?

You need to feel comfortable with math concepts such as the ones listed above. If you are not sure about some of them ("oh, I knew that...!"), but are ready to refresh your memory using external Web resources such as Wikipedia, then make an attempt and visit the course. You will probably get most of it. Anyway, this is not a math course!

I already know a lot about Algorithmic Information. Will I learn something new?

Yes, definitely. This course includes original content. You may know mathematical aspects of AIT, but not its applications. Or the converse, you know for instance how to apply "MDL", but may not be fully aware of the underlying theoretical motivations. And you will be amazed to see how AIT applies to human intelligence!

¿Quién puede hacer este curso?

Lamentablemente, las personas residentes en uno o más de los siguientes países o regiones no podrán registrarse para este curso: Irán, Cuba y la región de Crimea en Ucrania. Si bien edX consiguió licencias de la Oficina de Control de Activos Extranjeros de los EE. UU. (U.S. Office of Foreign Assets Control, OFAC) para ofrecer nuestros cursos a personas en estos países y regiones, las licencias que hemos recibido no son lo suficientemente amplias como para permitirnos dictar este curso en todas las ubicaciones. edX lamenta profundamente que las sanciones estadounidenses impidan que ofrezcamos todos nuestros cursos a cualquier persona, sin importar dónde viva.

¿Te interesa este curso para tu negocio o equipo?

Capacita a tus empleados en los temas más solicitados con edX para Negocios.