July 1st, 2020

Закон сохранения информации

Компьютер не создаёт новой информации,
он лишь осуществляет полезную трасформацию уже имеющейся информации.

Леон Бриллюэн, Science and Information Theory, 1956.

Что такое информация и почему она в среднем не возрастает натуралистически (то есть за счёт комбинированного действия случайных и закономерных факторов)?

1. Неформальное введение: Eric Holloway: Why is Bell's theorem important for conservation of information?

2. William Dembski, Robert Marks: Conservation of Information in Search: Measuring the Cost of Success, IEEE Trans. on Systems, Man and Cybernetics, Part A, Vol.39(5), Sept.2009, 1051-1062.

Conservation of information theorems indicate that any search algorithm performs, on average, as well as random search without replacement unless it takes advantage of problem-specific information about the search target or the search-space structure. Combinatorics shows that even a mod-erately sized search requires problem-specific information to be successful. Computers, despite their speed in performing queries, are completely inadequate for resolving even moderately sized search problems without accurate information to guide them. We propose three measures to characterize the information requiredfor successful search: 1) endogenous information, which measuresthe difficulty of finding a target using random search; 2) exogenous information, which measures the difficulty that remainsin finding a target once a search takes advantage of problem-specific information; and 3) active information, which, as the difference between endogenous and exogenous information, measures the contribution of problem-specific information for successfully finding a target. This paper develops a methodology based on these information measures to gauge the effectiveness with which problem-specific information facilitates successful search. It then applies this methodology to various search tools widely used in evolutionary search.

3. William Dembski, Robert Marks: The search for a search: Measuring the Information Cost of Higher Level Search, Journal of Advanced Computational Intelligence and Intelligent Informatics (JACIII), Volume 14, 2010, 475-486.

Needle-in-the-haystack problems look for small targets in large spaces. In such cases, blind search stands no hope of success. Conservation of information dictates any search technique will work, on average, as well as blind search. Success requires an assisted search. But whence the assistance required for a search to be successful? To pose the question this way suggests that successful searches do not emerge spontaneously but need themselves to be discovered via a search. The question then naturally arises whether such a higher-level “search for a search” is any easier than the original search. We prove two results: (1) The Horizontal No Free Lunch Theorem, which shows that average relative performance of searches never exceeds unassisted or blind searches, and (2) The Vertical No Free Lunch Theorem, which shows that the difficulty of searching for a successful search increases exponentially with respect to the minimum allowable active information being sought.

Collapse )