No Access

INDEPENDENCE IN COMPLETE AND INCOMPLETE CAUSAL NETWORKS UNDER MAXIMUM ENTROPY

MICHAEL J. MARKHAM

Bradford University, 22 Wrenbury Crescent, Leeds LS16 7EG, UK

https://doi.org/10.1142/S0218488508005571Cited by:1 (Source: Crossref)

Abstract

In an expert system having a consistent set of linear constraints it is known that the Method of Tribus may be used to determine a probability distribution which exhibits maximised entropy. The method is extended here to include independence constraints (Accommodation).

The paper proceeds to discusses this extension, and its limitations, then goes on to advance a technique for determining a small set of independencies which can be added to the linear constraints required in a particular representation of an expert system called a causal network, so that the Maximum Entropy and Causal Networks methodologies give matching distributions (Emulation). This technique may also be applied in cases where no initial independencies are given and the linear constraints are incomplete, in order to provide an optimal ME fill-in for the missing information.

Keywords:

References

M. Tribus , Rational Descriptions, Decisions and Designs ( Pergamon , New York , 1969 ) . Google Scholar
R. E. Neapolitan , Probabilistic Reasoning in Expert Systems: Theory and Algorithms ( John Wiley & Sons , N.York , 1990 ) . Google Scholar
J. Pearl, On filling in missing information in causal networks. Submitted to Knowledge-based Systems (1986) . Google Scholar
L. de Campos and S. Moral, Independence concepts for convex sets of probabilities, Proc. 11th Conf. Uncertainty in AI (Morgan Kaufmann, 1995) pp. 108–115. Google Scholar
P. Wally , Statistical Reasoning with Imprecise Probabilities ( Chapman and Hall , New York , 1991 ) . Crossref, Google Scholar
G. F. Cooper, Probabilistic Inference Using Belief Networks is NP-Hard, Technical Report KSL-87-27 (Stanford University, California, 1988) . Google Scholar
I. Maung and J. B. Paris, Int. J. Intelligent Systems 5(5), 595 (1990), DOI: 10.1002/int.4550050509. Crossref, Web of Science, Google Scholar
J. B. Paris and A. Vencovská, Int. J. Approximate Reasoning 4(3), 183 (1990), DOI: 10.1016/0888-613X(90)90020-3. Crossref, Google Scholar
J. B. Paris , The Uncertain Reasoner's Companion ( Cambridge University Press , 1994 ) . Google Scholar
J. B. Paris, Artif. Intell. 29, (2003). Google Scholar
G. R. Garside and P. C. Rhodes, Knowledge-Based Systems 9, 315 (1996), DOI: 10.1016/0950-7051(96)01039-8. Crossref, Web of Science, Google Scholar
G. R. Garside, P. C. Rhodes and D. E. Holmes, Knowledge-Based Systems 12(3), 101 (1999), DOI: 10.1016/S0950-7051(99)00002-7. Crossref, Web of Science, Google Scholar
P. C. Rhodes and G. R. Garside, Knowledge-Based Systems 10(4), 213 (1998), DOI: 10.1016/S0950-7051(97)00033-6. Crossref, Web of Science, Google Scholar
D. E. Holmes and P. C. Rhodes, Int. J. Intelligent Systems 13(9), 841 (1998). Crossref, Web of Science, Google Scholar
D. E. Holmes, P. C. Rhodes and G. R. Garside, Int. J. Intelligent Systems 14(6), 535 (1999). Crossref, Web of Science, Google Scholar
D. E. Holmes, Independence relationships implied by d-separation in the Bayesian model of a causal tree are preserved by the maximum entropy model, Bayesian Inference and ME Methods in Science and Engineering (AIP Conf. Proc., Vol. 567, Melville, New York, 2001) . Google Scholar
J. Williamson, Foundations for Bayesian networks, Foundations of Bayesianism, Kluwer Applied Logic Series (2001) pp. 75–115. Crossref, Google Scholar
J. Williamson, Linköping Electronic Articles in Computer and Information Science 7(0), 1 (2002). Web of Science, Google Scholar
M. J. Markham and P. C. Rhodes, Int. J. Uncertainty, Fuzziness and K-BS 7(1), 63 (1999), DOI: 10.1142/S0218488599000052. Link, Web of Science, Google Scholar
G. Kern-Isberner, Artif. Intell. 98(1–2), 169 (1998), DOI: 10.1016/S0004-3702(97)00068-4. Crossref, Web of Science, Google Scholar
M. J. Markham, Probabilistic Independence and its Impact on the Use of Maximum Entropy in Causal Networks, PhD. Thesis (School of Informatics, Dept. of Computing, University of Bradford, 2005) . Google Scholar
C. E. Shannon and W. Weaver , A Mathematical Theory of Communication ( University of Illinois , 1964 ) . Google Scholar
D. Geiger, T. S. Verma and J. Pearl, d-Separation: From theorems to algorithms, Uncertainty in AI(5) (North-Holland, 1990), pp. 139–148 . Google Scholar
S. Acid and L. de Campos, An algorithm for finding minimum d-separating sets in belief networks, Proc. 12th Conf. Uncertainty in AI (Morgan Kaufman, San Mateo, California, 1996) pp. 3–10. Google Scholar
T. S. Verma and J. Pearl, Causal networks: Semantics and expressiveness, Proc. 4th AAAI Workshop on Uncertainty in AI (North-Holland, 1988) pp. 352–359. Google Scholar
D. Geiger and J. Pearl, On the logic of causal models, Proc. 4th AAAI Workshop on Uncertainty in AI (1988) pp. 136–147. Google Scholar
T. Lukasiewicz, Credal networks under maximum entropy, Proc. 16th Conf. Uncertainty in AI (2000) pp. 363–370. Google Scholar
S. L. Lauritzen and D. J. Spiegelhalter, J. Royal Statistical Society B 50(2), 157 (1988). Google Scholar