Mathematics for Computer Science What is this course about? Lecture 1: Overview and review of basic Set Theory I There are many things that a course called “Mathematics for Computer Science” could be about. It could be a course in discrete mathematics, the tool that helps us understand data structures and algorithmics. Or it could be a course in probability theory, information theory, and statistics, topics which are useful for many applications. I But this is a course that deals with the concepts of computability, uncomputability and computational complexity. That is, this course is not just a course in applied mathematics — rather, its aim is to introduce the kind of mathematics that tell us what computer science actually is. Areces, Blackburn and Figueira TALARIS team INRIA Nancy Grand Est Contact: [email protected] Course website: http://www.loria.fr/~blackbur/courses/math Areces, Blackburn, Figueira: Mathematics for Computer Science Turing machine I I Church’s thesis The central concept in this course is the concept of a Turing machine. This is a very abstract — and incredibly simple — model of what computation is. It is also a very natural and plausible notion of computation, and was inspired by reflecting on what people do when they actually calculate something. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Part of the answer is obvious: if we have a robust model of what computation is (and Turing machines are believed to be such a model) then we have a mathematical basis for computer science. That is, the model gives us lots of positive information about computability. I Moreover, the concept of a Turing machine splits the mathematical world into two. A consequence of the model is that there are many well-defined mathematical problems that cannot be computed by any means. That is, the model also gives us lots of negative information about computability. c INRIA Church’s thesis says that anything at all that we can compute can be computed by a Turing machine! I That is, this simple model appears to provide a universal model of computation. I And indeed, over the years, many different models of computation have been proposed. But they have all turned out to be equivalent to the notion of a Turing machine. c INRIA I But these concepts have ramifications way beyond the confines of theoretical computer science — they are also relevant to such domains as logic, philosophy, cognitive science, and artificial intelligence. I The link with logic is deep and intimate. In fact, it was logicians such as Kurt G¨ odel, Alonzo Church, Alan Turing, and Stephen Kleene who first introduced and explored the concept of computability. The concept grew naturally out of their logical investigations — and in turn fed back into logic and revealed its limitations. c INRIA Impact on Cognitive Science and Artificial Intelligence The impact on philosophy has also been profound. The idea that certain problems are not computable (that is, not solvable by any realistic computational device) — and provably not solvable — was radical at the time, and affected Anglo-American philosophical culture deeply. But the area raises many other interesting philosophical issues. To name just one: what is the status of Church’s thesis? It is not a mathematical truth, something that can be proved. But it is not obviously correct to call it an empirical generalisation either. So what is its status? Areces, Blackburn, Figueira: Mathematics for Computer Science I Areces, Blackburn, Figueira: Mathematics for Computer Science Impact on Philosophy I As we shall see, at first sight, the concept of a Turing machine looks far too simple to be interesting. But here appearances are misleading. But it is also important for other reasons I I I Areces, Blackburn, Figueira: Mathematics for Computer Science Why is this material important? Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA c INRIA I Turing machines were inspired by thinking about the process of calculation; they have at least some (abstract) claim to cognitive plausibility. I Moreover the model they gave rise to — devices with internal states — provided exactly the kinds of ideas that helped psychology to break free of behaviorism in a scientific way, thus giving rise to modern cognitive science. I It also suggested that intelligence was something that could not only be modeled, but perhaps even created by programming. This set the scene for the birth of artificial intelligence. Areces, Blackburn, Figueira: Mathematics for Computer Science 1 c INRIA Linking these ideas together another way Alan Turing (1912–1954) I The ideas underlying the course can be tied together in another way — by considering the career of one man, namely Alan Turing. I It may seem bold to claim that the ideas underlying Turing Machines touch on areas as diverse as Philosophy and Computer Science. I But Turing himself would not have been surprised: indeed, he made many of the connections himself. I So who was Alan Turing . . . ? Areces, Blackburn, Figueira: Mathematics for Computer Science Alan Turing (born 23 June 1912, died 7 June 1954) was a British logician, cryptographer, a key figure in the birth of theoretical and practical computer science, and a pioneer of Artificial Intelligence and Cognitive Science (with special reference to Natural Language). He also did important early work on the formation of patterns and mathematical biology. c INRIA Areces, Blackburn, Figueira: Mathematics for Computer Science Alan Turing c INRIA Early life I He excelled in science — though this was at a time when science was not in such high regard as it is now. Because he couldn’t be bothered studying Latin or Greek intensively, he failed to win a scholarship to Trinity College Cambridge and went to King’s College Cambridge instead. I He was an undergraduate there from 1931–1934, and in 1935 was elected to be fellow of the college because of the strength of his mathematical research. Born in London, Turing genius was recognized by many of his teachers from the age of 6 onwards. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Areces, Blackburn, Figueira: Mathematics for Computer Science Computability and Incompatibility I I I I World War II and Cryptography In his classic 1936 paper “On Computable Numbers, with an Application to the Entscheidungsproblem” Turing introduced what are now called Turing machines, simple formal devices which naturally modeled computation. He showed convincingly that such machines would be capable of performing any conceivable mathematical problem that could be dealt with algorithmically. These simple machines are still a key concept in the study of computability — and indeed computational complexity. In 1937 and 1938 he was in Princeton, working under Alonso Church. He gained his PhD there in 1938; his thesis dealt with even more abstract notions of computation, namely Turing machines equipped with Oracles (a concept we won’t discuss in this course). Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA I During the Second World War Turing was the key figure in the British attempts at Benchley Park to break German codes, and in particular, the codes used by U-boats. I His work (which was classified for many years after the war) was crucial to Britain’s survival; it these codes had not been broken it seems likely that Britain would have been forced to negotiate a treaty with Germany. Areces, Blackburn, Figueira: Mathematics for Computer Science Enigma Machine c INRIA Turing’s response I Each German U-boat was equipped with an Enigma machine, capable of encoding and decoding transmissions. I Eventually an Enigma machine was captured — but how could it be understood? U-boat Enigma’s could have up to 1022 states! Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA c INRIA I His answer was the creation of the Bombe. This was an extension of an earlier Polish-designed hand-operated code breaking machine called Bomba; apparently when a possible solution was reached, a part would fall onto the floor, making a noise — and hence the name! I The Bombe was, in effect, an early computer, though not a universal computer, that could rapidly test many possible settings of the Enigma machine. It used logical deduction to rule out certain settings as impossible. Areces, Blackburn, Figueira: Mathematics for Computer Science 2 c INRIA The Bombe The postwar development of the computer I After the war Turing worked at the National Physical Laboratory. I In 1946 he designed one of the first stored-program computers (that is, one of the first real-existing universal machines). The Bombe was a very real machine — and a very useful one — not an abstraction like his Turing Machines. Turing had the first Bombe running by March 1940, just 6 months after he went to Benchley. By the end of the war, 200 were in operation. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Areces, Blackburn, Figueira: Mathematics for Computer Science Pilot version of ACE Pilot ACE (reconstruction, Science Museum London) I I A full version of the machine, which was called the ACE, was never actually made, though a smaller pilot version was. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA c INRIA I Further pioneering work in theoretical computer science: for example, the concept of compilation — translating a higher level programming language down into a machine language. Though legend has it that Turing thought compilation was unnecessary — after all, it was so easy to compute directly in binary code! I Pioneering work in AI: Turing also wrote the first chess playing program. Unfortunately, no computer powerful enough to run it actually existed at the time. However he did simulate it by hand (taking half an hour to compute each move) and the program is said to have won some games. Areces, Blackburn, Figueira: Mathematics for Computer Science The Turing test I I c INRIA Legacy The test was pioneering concept in the philosophy of AI and Cognitive Science; it was introduced in his classic 1950 paper “Computing Machinery and Intelligence”. In essence, he replaced the question “Can a machine think?” by a concrete experiment: if a program can chat with a person (in the modern sense of using MSN, or Yahoo or Skype . . . ) over an extended session, and at the end of the session the person cannot decide if he was chatting with a computer or a machine, then we must conclude we have a machine that thinks. Turing concluded: I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Other work from this period The Manchester Mark 1 was one of the world’s first fully general computers (that is, it was a physical embodiment of the ideas underlying the abstract notion of a Turing machine). I A pilot version of ACE ran its first program on 10/5/1950. But by this time, Turing was no longer at the National physical laboratory, because in 1948 he moved to the Computer Science Department of Manchester University where he participated in the development of the Manchester Mark 1. Areces, Blackburn, Figueira: Mathematics for Computer Science Manchester Mark 1 - 1949 Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA c INRIA I Turing died in 1954, probably by suicide. I He has been widely recognized and honoured since his death. For example, the computer science world’s equivalent of the the Nobel Prize (awarded by the Association for Computing Machinery) is called the Turing Prize. I He was in Time Magazine’s list of 100 most influential people of the 20th century. I And much else beside . . . Areces, Blackburn, Figueira: Mathematics for Computer Science 3 c INRIA Marathon Man And now, back to the course . . . What will you get out of this course? An understanding of the most fundamental concepts of theoretical computer science, namely that there are both computable and uncomputable functions. You will gain deeper insight into what computation actually is, and see how simply the concept can be modeled. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Areces, Blackburn, Figueira: Mathematics for Computer Science And now, back to the course . . . And now, back to the course . . . What will you get out of this course? What will you get out of this course? You will gain certain concrete mathematical skills. In particular, we will be paying close attention to how we prove things, that is, the way we can show general facts about the concepts we introduce. At the end of the course you will have seen a number of proofs, often of a very different nature. In short: this course should develop your abstract reasoning skills. Areces, Blackburn, Figueira: Mathematics for Computer Science You will have seen some deep mathematical logical results. The ideas discussed in this course are closely linked with key concepts of modern logic. We will show you these links, and this should deepen and enrich your understanding of logic. c INRIA Areces, Blackburn, Figueira: Mathematics for Computer Science And now, back to the course . . . And, although it is not the primary aim of the course, we hope you will also gain deeper insight into how the concept of computability touches on cognitive science and natural language. That is, this course should give you deeper insight into Alan Turing’s legacy. c INRIA I Lecture 1 Overview and review of basic set theory. I Lecture 2 : Functions and equinumerous sets. I Lecture 3 : Turing machines. I Lecture 4 : Church’s thesis. I Lecture 5 : Uncomputability and undecidability I. I Lecture 6 : Uncomputability and undecidability II. I Lecture 7 : Undecidability of first-order logic. I Lecture 8 : Computational complexity I. I Lecture 9 : Computational complexity II. I Lecture 10 : Computational complexity III. Areces, Blackburn, Figueira: Mathematics for Computer Science And what about in the rest of this lecture? c INRIA Basic concepts of set theory In the rest of this first lecture we will be introducing/revising the basic ideas of Set Theory. Nowadays set theory is a setting in which almost all mathematics is conducted. A quick review of basic set theory will give us the mathematical background we need for the course — and will lead us to interesting (and surprising) territory very fast. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Lectures What will you get out of this course? Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA c INRIA I A set is a collection of ‘things’ — the ‘things’ might be numbers, concrete objects, or other sets. I The items in a set are called its elements or its members. I If x is an element of a set y we say that “x is an element of y” or “x is a member of y” and write x ∈ y. I When are two sets equal? Whenever they contain exactly the same elements. I To put it the other way around: two sets are different if and only if they contain at least one different element. I If set x and set y are equal we write x = y. I You will often hear it said that sets are extensional. This simply means that they are completely determined by their elements. Areces, Blackburn, Figueira: Mathematics for Computer Science 4 c INRIA Specifying (finite) sets The Empty Set I We specify a set (that is, we say what set we are talking about) by saying what its elements are. I A very useful set is the empty set; this is the set that contains no members at all. I For example, a perfectly good set is the set containing the the first three positive integers 1, 2, and 3. I We usually write ∅ (or { }) for the empty set. I Why is the empty set “useful”? It may seem a bit strange. In fact it is useful in much the same way that the number 0 is useful in mathematics — it helps give sets an elegant algebraic structure (it completes the picture, so to speak). We’ll see some examples soon. I The empty set is unique. That is, there is only one empty set. Why is this? I We could write this set as {1, 2, 3}. I That is, we can specify a set by listing its elements and enclosing them in braces, that is, the symbols { and }. I Of course, this method only works for finite sets (and indeed, is only practical for small finite sets). As we shall see soon, there are other ways of specifying sets. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Areces, Blackburn, Figueira: Mathematics for Computer Science Subset I I I I I I c INRIA Proper subset A set x is a subset of a set y if every element of set x is an element of set y. The notation x ⊆ y means that x is a subset of y. It follows from this definition that every set is a subset of itself. That is, for any set x we have that if x ⊆ x. So ⊆ is a reflexive relation between sets. It also follows from this definition that for any sets x , y and z, if x ⊆ y and y ⊆ z then x ⊆ z. That is, ⊆ is a transitive relation between sets. It also follows from this definition that for any sets x and y, that if x ⊆ y and y ⊆ x then x = y. That is, ⊆ is a antisymmetric relation between sets. Do not get confused between ∈ and ⊂! They are very different concepts! Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Intersection is an operation that takes two sets and makes a new one; we use ∩ to indicate this operation. I We define this operation as follows: for any sets x and y, x ∩ y is the set containing precisely those elements that belong to x and y. I If x ∩ y = ∅ then we say that x and y are disjoint. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA A set x is a proper subset of y if x is a subset of y but x is not equal to y. I This is usually written as x ( y. c INRIA I Union is also an operation that takes two sets and makes a new one; we use ∪ to indicate this operation. I We define this operation as follows: for any sets x and y, x ∪ y is the set containing precisely those elements that belong to either x or y. Areces, Blackburn, Figueira: Mathematics for Computer Science Difference c INRIA Algebra of sets (I) These operations induce a rich algebraic structure on sets. For example, we have that: I Difference is another operation that takes two sets and makes a new one; we use \ to indicate this operation. I We define this operation as follows: for any sets x and y, x\y is the set containing precisely those elements of x that do not belong to y. Areces, Blackburn, Figueira: Mathematics for Computer Science I Union I Suppose x and y do not have any elements in common. Then x ∩ y = ∅. That is, as promised, ∅ is playing and algebraic role similar to that of the number 0. Sometimes we speak of one set being a proper subset of another. Areces, Blackburn, Figueira: Mathematics for Computer Science Intersection I I c INRIA I Commutative rules: x∪y =y∪x x∩y =y∩x I Associative rules: x ∪ (y ∪ z) = (x ∪ y) ∪ z x ∩ (y ∩ z) = (x ∩ y) ∩ z Areces, Blackburn, Figueira: Mathematics for Computer Science 5 c INRIA Algebra of sets (II) Sets of numbers Sets of numbers play an important role in mathematics, and will play an important role in this course. Two important sets of numbers are: These operations induce a rich algebraic structure on sets. For example, we have that: I I Distributive rules: x ∪ (y ∩ z) = (x ∪ y) ∩ (x ∪ z) x ∩ (y ∪ z) = (x ∩ y) ∪ (x ∩ z) I The set of natural numbers is the set whose elements are the numbers 0, 1, 2, 3, 4, 5, . . . and so on. We use N (or ω) to specify this (infinite) set. I The set of integers is the set whose elements are the numbers . . . −5, −4, −3, −2, −1, 0, 1, 2, 3, 4, 5, . . . and so on. We use Z to specify this set. Obviously Z is an infinite set, and obviously N ⊆ Z. De Morgan rules: x\(y ∪ z) = (x\y) ∩ (x\z) x\(y ∩ z) = (x\y) ∪ (x\z) Remark: in set theory, N is usually taken to be the fundamental infinite set, and other sets of numbers are defined in terms of it. For more information on this topic, see the references at the end of the lecture. Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Areces, Blackburn, Figueira: Mathematics for Computer Science More sets of numbers I I c INRIA Specifying richer sets II The set of positive integers is the set whose elements are the numbers 1, 2, 3, 4, 5, . . . and so on. We use Z+ to specify this set. Clearly Z+ = N\{0}. I The set of negative integers is the set whose elements are the numbers −1, −2, −3, −4, −5, . . . and so on. We use Z− to specify this set. Clearly Z = Z− ∪ {0} ∪ Z+ . The simple “list and enclose in braces method” only works for finite sets. What about infinite sets, like the set of all even numbers? I We specify these sets by describing them. For example, the set of even numbers is the set I The set of rational numbers (or fractions) is the set of all numbers of the from n/m where n ∈ Z and m ∈ Z+ . We use Q as a name of this set. I The set of real numbers is the set of all numbers standardly used in a calculus course. It consists of Q together with all the transcendental numbers such as π and e. We use R as a name for this set. Areces, Blackburn, Figueira: Mathematics for Computer Science {x ∈ N : for some y, 2 × y = x} I c INRIA Areces, Blackburn, Figueira: Mathematics for Computer Science Power Set I For any set x, we are free to form another set called the power set of x. I The power set of a set x is defined to be the set of all subsets of x. I For any set x, we denote the power set of x by P(x). I For any set x, both x and ∅ belong to P(x). I What is the powerset of ∅? Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA How large are power sets? As we shall see, power sets are very interesting. Let’s ask a very basic question about them, namely how large are (finite) power sets? Here is the answer: If a finite set contains n elements, then its power set contain 2n elements. c INRIA I As we shall learn, this is a more important result than it at first appears. I But that’s for the future. A more basic question is: how do we prove it. We shall do so using a fundamental proof technique: mathematical induction. Areces, Blackburn, Figueira: Mathematics for Computer Science Mathematical induction c INRIA It’s just like knocking over dominoes . . . I Many mathematical problems ask us to show that something is true for all natural number n. I We can prove this as follows. First show that the result holds for 0. This is called proving the base case. I Then show that if the result holds for some natural number n > 0, it also holds for n + 1. This is called proving the inductive step. I If we can do these two things, we have shown that the result holds for all natural numbers, and proved the result. I I Areces, Blackburn, Figueira: Mathematics for Computer Science There are important restrictions on what the description can be; see the references at the end of the lecture for further discussion. c INRIA Proving the base case is like knocking over the first domino. Proving the inductive step is like showing the dominoes are spaced so that each will hit the next. Areces, Blackburn, Figueira: Mathematics for Computer Science 6 c INRIA The power set example: base case The power set example: inductive step I I We want to prove that for every natural number n, if a set x has n elements then its power set has size 2n . I So let’s consider the smallest finite set of all, the empty set, the one containing 0 elements. We want to show that its power set contains 1 element, I But we know that this is true. After all, P(∅) is {∅}. And this has 1 element, which is what we wanted to show. I So we have proved the base case. I I I I I Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA Areces, Blackburn, Figueira: Mathematics for Computer Science Summary of Lecture 1 Key concepts: c INRIA Mathematical induction is a proof technique used to show that some property holds for all natural numbers n. Mathematical induction consists of showing two things: I I First, that the desired property holds for 0 (the base case). Second, that if the property holds for n, it also holds for n + 1 (the inductive step). Areces, Blackburn, Figueira: Mathematics for Computer Science c INRIA I We specify a set by saying what its elements are. For finite sets, we can do so simply by listing them. For more complex sets, we can describe how they are constructed out of sets that we know about, using ∪, ∩, \ and P (that is, by forming power sets. I There are many “standard” mathematical sets, and today we met some of them. The simplest (and smallest) of these was ∅. We also met a number of standard infinite sets of numbers, namely N, Z, Z+ , Z− , Q, and R. I Elements of Set Theory, by Herbert Enderton, Academic Press 1977. One of the most readable introductions to Set Theory. Clear, accurate, and covers quite a lot of ground. I Naive Set Theory, by Paul Halmos, Springer, 1970. This slim book is a classic mathematics text. It covers a lot of ground with a minimum of notation. I The Wikipedia article on mathematical induction is quite useful; it considers some variants of mathematical induction, and has some interesting historical remarks. I Alan Turing: The Enigma of Intelligence, by Andre Hodges, Burnett Books, 1983. A readable and interesting biography of Alan Turing. I A History of Modern Computing, by Paul Ceruzzi, MIT Press, 1983. An excellent source. I The Wikipedia article on Alan Turing is also quite informative and has useful links to related topics. Areces, Blackburn, Figueira: Mathematics for Computer Science References on Set Theory Areces, Blackburn, Figueira: Mathematics for Computer Science Sets are a key mathematical notion. A set is determined by its elements. Two sets are equal if and only if they have the same elements. c INRIA References on Alan Turing We also made use of a key proof technique, namely mathematical induction. I I Areces, Blackburn, Figueira: Mathematics for Computer Science Summary of Lecture 1 (continued) I c INRIA Summary of Lecture 1 (continued) Key concepts: I We briefly mentioned some of the key concepts we will be discussing in this course, such as Turing machines and Church’s Thesis. I Although we did not explain these concepts in detail we explained why they are of such fundamental importance to computer science: they give a simple model of what it means for something to be computable and (just as importantly) show that many things are not computable. I We also indicated the importance of these topics for other subjects than computer science, such as Cognitive Science, Philosophy, Artificial Intelligence and Natural language Processing. I Indeed, as we saw, many of these connections were introduced and explored by Alan Turing himself. Areces, Blackburn, Figueira: Mathematics for Computer Science Now let us suppose that for some natural number n ≥ 0 we have shown that for any set with n elements, its power set has 2n elements. We want to show that this implies that shown that for any set with n + 1 elements, its power set has 2n+1 elements. So let S be any set containing n + 1 elements. Let’s write it as {s1 , s2 , s3 , . . . , sn , sn+1 }. Or to put it another way, S = T ∪ {sn+1 }, where T is {s1 , s2 , s3 , . . . , sn } Now the power set of T contains 2n elements. Why?. And now for the key observation: every element in the power set of S is either an element p of the power set of T , or and has the form p ∪ {sn+1 }. Why? So there are 2n + 2n elements in P(S). But 2n + 2n = 2 × 2n = 2n+1 , which is what we wanted to show. c INRIA 7 c INRIA
© Copyright 2024