Mark Sam Miller
Mark S Miller
Mark S. Miller is an American computer scientist. He is known for his work as one of the participants in the 1979 hypertext project known as Project Xanadu; for inventing Miller columns; as the co-creator of the Agoric Paradigm[1] of market-based distributed secure computing; and the open-source coordinator of the E programming language. He also designed the Caja programming language.
Miller earned a BS in computer science from Yale in 1980 and published his Johns Hopkins PhD thesis in 2006.[2] Previously Chief Architect with the Virus-Safe Computing Initiative at HP Labs, he is now a research scientist at Google[3] and a member of the ECMAScript (JavaScript) committee.[4] - Wikipedia
- twitter.com/marksammiller
-
The Agoric Papers
These three papers by Mark S. Miller and K. Eric Drexler appeared in The Ecology of Computation, Bernardo Huberman (ed.) Elsevier Science Publishers/North-Holland, 1988.
- Mark Miller: Agoric and the Decades-Long Quest for Secure Smart Contracts. -Epicenter Podcast
Dissertation
-
Robust Composition: Towards a Unified Approach to Access Control and Concurrency Control by Mark Samuel Miller
- A dissertation submitted to Johns Hopkins University in conformity with the requirements for the degree of Doctor of Philosophy.
27.1 Contributions
This dissertation makes seven primary contributions:
- The object-capability model, a new model of secure computation that is abstract enough to describe both prior object-capability languages and operating systems, and concrete enough to describe authority-manipulating behavior.
- A novel approach to concurrency control in terms of attenuating authority, including a unified architecture in which both access control and concurrency control concerns may be addressed together.
- A clearly framed distinction between “permission” and “authority.” We develop a taxonomy of computable bounds on eventual permission and authority, enabling us to reason about many simple access abstractions. Using this framework, we provide a unified account of access control and authority in distributed systems consisting of object-capability islands sending messages to each other over a cryptographic capability sea.
- A system of reference states, transition rules, and message delivery properties associated with each state, that reify partial failures in a manner that permits robust application-level recovery. In E, these are coupled to language constructs that enable programmers to anticipate and handle distributed plan failures at the syntactic locus where a dependency on remote behavior is introduced. These same syntactic constructs alert the programmer of the loci where interleavings of execution may occur.
- A generalization of prior work on promise pipelining from asymmetric client-serversystems to symmetric peer-to-peer systems. Promises enable explicit expression ofdataflow in distributed computation. Promise pipelining enables this computationalstructure to be exploited without cascading round trips.
- An adaptation of non-signaling errors to the propagation of broken references, allowingprogrammers to consistently defer handling of reference failure when direct defense is inappropriate, and to compose chains of data-dependent computation whose referencefailure propagation has a sensible semantics. By providing a coherent error reporting behavior for promise pipelines, this mechanism provides a manageable foundation forrecovery from delayed errors in distributed dataflow computations.
- Finally, we introduce E-ORDER, a message delivery ordering constraint that achieves an enforceable middle ground between CAUSAL and FIFO message orderings. The resulting ordering is strong enough to prevent adversaries from exploiting race conditions to obtain inappropriate access, but weak enough to be enforceable. We show how the delivery ordering may be defined in terms of an attenuation of the authority provided by passed references. This allows us to reason about the effects of messaging within our unified model of concurrency and access control.
Several programming systems described in Chapter 26 have adopted (in varying subsets) each of the contributions noted above by borrowing elements from the E system. Twisted Python, in particular, is reported to be in use in more than 50 real-world applications at the time of this writing.
Bibliography
- [AW04] Mart ́ın Abadi and Ted Wobber. A Logical Account of NGSCB. InProc. For-mal Techniques for Networked and Distributed Systems – Forte 2004. Springer-Verlag, Berlin Germany, September 2004.
- [Bej96] Arturo Bejar. The Electric Communities Distributed Garbage Collector,1996. www.crockford.com/ec/dgc.html.
- [BH77] Henry G. Baker, Jr. and Carl E. Hewitt. The Incremental Garbage Collectionof Processes. InProc. International Joint Conference on Artificial Intelligence,pages 55–59, Cambridge, Massachusetts, August 1977. IJCAI.
- [Bir05] Kenneth P. Birman.Reliable Distributed Systems. Springer Verlag, 2005.
- [BJ87] Ken Birman and T. Joseph. Exploiting Virtual Synchrony in Distributed Sys-tems. InSOSP ’87: Proc. Eleventh ACM Symposium on Operating SystemsPrinciples, pages 123–138, New York, NY, USA, 1987. ACM Press.
- [BL94] Phillip Bogle and Barbara Liskov. Reducing Cross Domain Call OverheadUsing Batched Futures. InOOPSLA ’94: Proc. Ninth Annual Conference onObject-Oriented Programming Systems, Language, and Applications, pages341–354, New York, NY, USA, 1994. ACM Press.
- [Bla85] Andrew P. Black. Supporting Distributed Applications: Experience withEden. InProc. Tenth Annual Symposium on Operating Systems Principles,pages 181–193. ACM, December 1985.
- [BNOW94] Andrew Birrell, Greg Nelson, Susan Owicki, and Edward Wobber. NetworkObjects. InProc. Fourteenth ACM Symposium on Operating Systems Princi-ples, pages 217–230, Asheville, NC, 1994. ACM Press.
- [Boe84] W. E. Boebert. On the Inability of an Unmodified Capability Machine to En-force the *-property. InProc. 7th DoD/NBS Computer Security Conference,pages 291–293, Gaithersburg, MD, USA, September 1984. National Bureauof Standards.
- [Boe03] Earl Boebert. Comments on Capability Myths Demolished, 2003. www.eros-os.org/pipermail/cap-talk/2003-March/001133.html.
- [Boe05] Hans-J. Boehm. Threads Cannot be Implemented as a Library.SIGPLANNotices, 40(6):261–268, 2005.
- [Bre73] T. H. Bredt. A Survey of Models for Parallel Computing. Technical ReportCSL-TR-70-8, Stanford University Computer Systems Laboratory, Stanford,CA, December 1973.
- [BS79] Matt Bishop and Lawrence Snyder. The Transfer of Information and Au-thority in a Protection System. InProc. 7th ACM Symposium on OperatingSystems Principles, pages 45–54, December 1979. Published as OperatingSystem Review, Vol 13, No 4.188
- [BST89] H. E. Bal, J. G. Steiner, and A. S. Tanenbaum. Programming Languages forDistributed Computing Systems.ACM Computing Surveys, 21(3):261–322,September 1989.
- [Car97] Luca Cardelli. Program Fragments, Linking, and Modularization. InThe 24thACM SIGPLAN-SIGACT Symposium on Principles of Programming Lan-guages, pages 266–277, Paris, January15–17 1997.
- [CC96] Antonio Cau and Pierre Collette. Parallel Composition of Assumption-Commitment Specifications: A Unifying Approach for Shared Variable andDistributed Message Passing Concurrency.Acta Informatica, 33(2):153–176,1996.
- [CDM01] A. Chander, D. Dean, and J. Mitchell. A State-Transition Model of TrustManagement and Access Control. In14th IEEE Computer Security Founda-tions Workshop, Cape Breton, Nova Scotia, June 2001.
- [CF91] R. Cartwright and M. Fagan. Soft Typing. InProc. SIGPLAN ’91 Conferenceon Programming Language Design and Implementation, pages 278–292, 1991.
- [Chu41] Alonzo Church.The Calculi of Lambda Conversion. Number 6 in Annals ofMathematical Studies. Princeton University Press, 1941.
- [CL85] K. Mani Chandy and Leslie Lamport. Distributed Snapshots: DeterminingGlobal States of Distributed Systems.ACM Transactions on Computer Sys-tems, 3(1):63–75, 1985.
- [CL02] Miguel Castro and Barbara Liskov. Practical Byzantine Fault Tolerance andProactive Recovery.ACM Transactions on Computer Systems, 20(4):398–461,2002.
- [Cli81] W. D. Clinger.Foundations of Actor Semantics. PhD thesis, Dept. of Math-ematics, Massachusetts Institute of Technology, Cambridge, MA, 1981.
- [Clo04a] Tyler Close. Decentralized Identification, 2004. www.waterken.com/dev/YURL/Definition/.
- [Clo04b] Tyler Close. Waterken YURL, 2004. www.waterken.com/dev/YURL/httpsy/.
- [Clo04c] Tyler Close. Web-calculus, 2004. www.waterken.com/dev/Web/.
- [Clo06] Tyler Close. Credit Transfer Within Market-based Resource Allocation In-frastructure. Tech Report HPL-2006-5, Hewlett Packard Laboratories, 2006.
- [Con67] Control Data Corporation.6400/6500/6600 Computer Systems; ReferenceManual. Control Data Corp., Documentation Dept., Palo Alto, CA, USA,revised May 1967 edition, 1967. ed-thelen.org/comp-hist/CDC-6600-R-M.html.
- [Cro97] Douglas Crockford, 1997. Personal communication.189
- [Cud99] Brian Cudahy.The Malbone Street Wreck. Fordham University Press, NewYork, 1999.
- [CW87] D. D. Clark and D. R. Wilson. A Comparison of Commercial and MilitaryComputer Security Policies. InProc. 1987 IEEE Symposium on Security andPrivacy, pages 184–195, 1987.
- [DA99] T. Dierks and C. Allen. The TLS protocol version 1.0, January 1999. InternetRFC 2246.
- [Den87] Daniel C. Dennett.The Intentional Stance. MIT Press, Cambridge, Mas-sachusetts, 1987.
- [DH65] J. B. Dennis and E. C. Van Horn. Programming Semantics for Multipro-grammed Computations. Technical Report MIT/LCS/TR-23, M.I.T. Labo-ratory for Computer Science, 1965.
- [Dij72] Edsger W. Dijkstra. The Humble Programmer.Communications of the ACM,15(10):859–866, 1972.
- [Dij76] Edsger W. Dijkstra.A Discipline of Programming. Prentice-Hall, EnglewoodCliffs, New Jersey, 1976.
- [DLP79] Richard A. DeMillo, Richard J. Lipton, and Alan J. Perlis. Social Pro-cesses and Proofs of Theorems and Programs.Communications of the ACM,22(5):271–280, 1979.
- [DN66] Ole-Johan Dahl and Kristen Nygaard. SIMULA - an ALGOL-based simula-tion language.Communications of the ACM, 9(9):671–678, 1966.
- [Don76] Jed Donnelley. A Distributed Capability Computing System, 1976.nersc.gov/∼jed/papers/DCCS/.
- [Don79] James E. Donnelley. Components of a Network Operating System.ComputerNetworks, 3:389–399, 1979.
- [DZK+02] Frank Dabek, Nickolai Zeldovich, Frans Kaashoek, David Mazieres, andRobert Morris. Event-driven Programming for Robust Software. InProc.10th ACM SIGOPS European Workshop, September 2002.
- [EA03] Dawson Engler and Ken Ashcraft. RacerX: Effective, Static Detection ofRace Conditions and Deadlocks. InProc. 19th ACM Symposium on OperatingSystems Principles, pages 237–252, Bolton Landing, NY, October 2003.
- [EFL+99] Carl Ellison, Bill Frantz, Butler Lampson, Ronald Rivest, B. Thomas, andT. Ylonen. SPKI Certificate Theory (IETF RFC 2693), September 1999.
- [Ell96] Carl Ellison. Establishing Identity Without Certification Authorities. InProc.Sixth USENIX Security Symposium, pages 67–76, Berkeley, 1996. Usenix.
- [Eng97] Robert Englander.Developing Java Beans. O’Reilly & Associates, Inc., 981Chestnut Street, Newton, MA 02164, USA, 1997
- [ES90] Margaret A. Ellis and Bjarne Stroustrup.The Annotated C++ ReferenceManual. Addison-Wesley, 1990.
- [Fab74] R. S. Fabry. Capability-based Addressing.Communications of the ACM,17(7):403–412, 1974.
- [FF02] Robert Bruce Findler and Matthias Felleisen. Contracts for higher-order func-tions.SIGPLAN Notices, 37(9):48–59, September 2002.
- [Fie00] Roy Thomas Fielding.Architectural Styles and the Design of Network-basedSoftware Architectures. PhD thesis, Dept. of Information and Computer Sci-ence, University of California at Irvine, 2000.
- [Fro03] A. Michael Froomkin. Toward a Critical Theory of Cyberspace.Harvard LawReview, 116(3):750–871, 2003. www.icannwatch.org/article.pl?sid=03/01/16/057208.
- [FW76a] Daniel P. Friedman and D. S. Wise. Cons Should not Evaluate its Argu-ments. In S. Michaelson and Robin Milner, editors,Automata, Languagesand Programming, pages 257–284. Edinburgh University Press, 1976.
- [FW76b] Daniel P. Friedman and David S. Wise. The impact of Applicative Program-ming on Multiprocessing. InProc. 1976 International Conference on Paral-lel Processing (ICPP’76), pages 263–272, Walden Woods, Michigan, August1976. IEEE.
- [GD72] G. Scott Graham and Peter J. Denning. Protection — Principles and Practice.Proc. Spring Joint Computer Conference, 40:417–429, 1972.
- [GHJV94] Erich Gamma, Richard Helm, Ralph Johnon, and John Vlissides. Design Patterns, Elements Of Reusable Object-Oriented Software. Addison Wesley,1994.
- [GK76] Adele Goldberg and Alan Kay. Smalltalk-72 Instruction Manual. TechnicalReport SSL 76-6, Learning Research Group, Xerox Palo Alto Research Center,1976.
- [GMPS97] L. Gong, M. Mueller, H. Prafullchandra, and R. Schemers. Going Beyondthe Sandbox: An Overview of the New Security Architecture in the JavaDevelopment Kit 1.2. InUSENIX Symposium on Internet Technologies andSystems, pages 103–112, Monterey, CA, 1997.
- [Gon89] Li Gong. A Secure Identity-based Capability System. InProc. 1989 IEEESymposium on Security and Privacy, pages 56–65, 1989.
- [Gon99] Li Gong.Inside Java 2 Platform Security. The Java Series. Addison Wesley,1999.
- [GR83] Adele Goldberg and David Robson.Smalltalk-80, The Language and its Im-plementation. Addison-Wesley, Reading, MA, 1983.
- [Gra73] Mark Granovetter. The Strength of Weak Ties.American Journal of Sociol-ogy, 78:1360–1380, 1973.
- [Gra86] Jim Gray. Why Do Computers Stop and What Can Be Done About It?InProc. 5th Symposium on Reliability in Distributed Software and DatabaseSystems, Los Angeles, CA, January 1986.
- [Gut04] Peter Gutmann.Cryptographic Security Architecture: Design and Verifica-tion. Springer Verlag, 2004.
- [Hal85] Robert H. Halstead, Jr. Multilisp: A Language for Concurrent SymbolicComputation.ACM Transactions on Programming Languages and Systems,7(4):501–538, October 1985.
- [Han93] Per Brinch Hansen. Monitors and Concurrent Pascal: a Personal History. InHOPL-II: The Second ACM SIGPLAN Conference on History of Program-ming Languages, pages 1–35, New York, NY, USA, 1993. ACM Press.
- [Har85] Norman Hardy. KeyKOS Architecture.SIGOPS Operating Systems Review,19(4):8–25, 1985.
- [Har88a] Norm Hardy. The Confused Deputy.Operating Systems Review, October1988.
- [Har88b] David Harel. On Visual Formalisms.Communications of the ACM, 31(5):514–530, 1988.
- [Hay37] Friedrich Hayek. Economics and Knowledge.Economica IV, pages 33–54,1937.
- [Hay45] Friedrich Hayek. The Uses of Knowledge in Society.American EconomicReview, 35:519–530, September 1945.
- [Hay64] Friedrich Hayek. The Theory of Complex Phenomena. In M Bunge, edi-tor,The Critical Approach to Science and Philosophy, pages 332–349. CollierMcMillan, London, 1964.
- [HB78] Carl Hewitt and Henry Baker. Actors and Continuous Functionals. In E. J.Neuhold, editor,Formal Description of Programming Concepts, pages 367–390. North-Holland, Amsterdam, 1978.
- [HBS73] Carl Hewitt, Peter Bishop, and Richard Steiger. A Universal Modular AC-TOR Formalism for Artificial Intelligence. In Nils J. Nilsson, editor, Proc. 3rd International Joint Conference on Artificial Intelligence, pages 235–245, Standford, CA, August 1973. William Kaufmann.
- [HDH03] Martin Hirzel, Amer Diwan, and Matthew Hertz. Connectivity-based GarbageCollection. InProc. ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA ’03), pages 359–373,2003.
- [Hew77] Carl Hewitt. Viewing Control Structures as Patterns of Passing Messages. Journal of Artificial Intelligence, 8(3):323–364, 1977.
- [Hew85] Carl Hewitt. The Challenge of Open Systems: Current Logic Programming Methods May be Insufficient for Developing the Intelligent Systems of theFuture. BYTE, 10(4):223–242, 1985.
- [Hoa65] C.A.R Hoare. Record Handling, in Algol Bulletin 21.3.6, 1965.
- [Hoa69] C. A. R. Hoare. An Axiomatic Basis for Computer Programming. Commu-nications of the ACM, 12(10):576–580, 583, October 1969.
- [Hoa74] C. A. R. Hoare. Monitors: an Operating System Structuring Concept. Communications of the ACM, 17(10):549–557, 1974.
- [Hoa78] C. A. R. Hoare. Communicating Sequential Processes. Communications of the ACM, 21(8):666–677, 1978.
- [HP97] Lenwood S. Heath and Sriram V. Pemmaraju. Stack and Queue Layoutsof Posets. SIJDM: SIAM Journal on Discrete Mathematics, 10(4):599–625, November 1997.
- [HPW03] Michael Howard, Jon Pincus, and Jeannette Wing. Measuring Relative Attack Surfaces. Technical Report CMU-TR-03-169, Carnegie Mellon University, August 2003. Proc. 2003 Workshop on Advanced Developments in Software and Systems Security. www.cs.cmu.edu/∼wing/publications/Howard-Wing03.pdf.
- [HRB+87] Norman C. Hutchinson, Rajindra K. Raj, Andrew P. Black, Henry M. Levy,and Eric Jul. The Emerald Programing Lanuage Report. Technical Report 87-10-07, Department of Computer Science, University of Washington, Seattle,October 1987.
- [HRU75] Michael A. Harrison, Walter L. Ruzzo, and Jeffrey D. Ullman. On Protectionin Operating Systems. InThe Symposium on Operating Systems Principles,pages 14–24, 1975.
- [HT91] Kohei Honda and Mario Tokoro. An Object Calculus for Asynchronous Communication. In European Conference on Object Oriented Programming, pages 133–147, 1991.
- [IBM68] IBM Corporation. IBM System/360 Principles of Operation. IBM Corpora-tion, San Jose, CA, USA, eighth edition, 1968.
- [IoEE85] American National Standards Institute, Institute of Electrical, and ElectronicEngineers. IEEE Standard for Binary Floating-Point Arithmetic.ANSI/IEEEStandard, Std 754-1985,New York, 1985.
- [JLS76] Anita K. Jones, Richard J. Lipton, and Lawrence Snyder. A Linear Time Algorithm for Deciding Security. InFoundations of Computer Science, pages 33–41, 1976.
- [Jon73] Anita K. Jones.Protection in Programmed Systems. PhD thesis, Departmentof Computer Science, Carnegie-Mellon University, June 1973.
- [Jon83] Cliff B. Jones. Specification and Design of (Parallel) Programs. InIFIPCongress, pages 321–332, 1983.
- [Jon03] Cliff B. Jones. The Early Search for Tractable Ways of Reasoning aboutPrograms.IEEE Annals of the History of Computing, 25(2):26–49, 2003.
- [Kah96] William Kahan. Lecture Notes on the Status of the IEEE Standard 754 forBinary Floating Point Arithmetic. www.cs.berkeley.EDU/∼wkahan/ieee754status/ieee754.ps, 1996.
- [Kar03] Alan H. Karp. Enforce POLA on Processes to Control Viruses. Communications of the ACM, 46(12):27–29, 2003.
- [Kay93] Alan C. Kay. The Early History Of Smalltalk.SIGPLAN Notices, 28(3):69–95, 1993.
- [Kay98] Alan Kay. Prototypes vs. Classes, 1998. lists.squeakfoundation.org/pipermail/squeak-dev/1998-October/017019.html.
- [KCR98] R. Kelsey, W. Clinger, and J. Rees. Revised5Report on the AlgorithmicLanguage Scheme.ACM Sigplan Notices, pages 26–76, 1998.
- [Key81] Key Logic, Inc. Gnosis Design Documentation, 1981. www.agorics.com/Library/KeyKos/Gnosis/keywelcome.html.
- [Key86] Key Logic, Inc.U.S. Patent 4,584,639: Computer Security System. U. S.Patent Office, 1986.
- [KGRB01] Alan H. Karp, Rajiv Gupta, Guillermo Rozas, and Arindam Banerji. The Client Utility Architecture: The Precursor to E-Speak. Technical Report HPL-2001-136, Hewlett Packard Laboratories, June 09 2001.
- [KGRB03] Alan H. Karp, R. Gupta, G. J. Rozas, and A. Banerji. Using Split Capabilitiesfor Access Control.IEEE Software, 20(1):42–49, January 2003.
- [KH84] P. A. Karger and A. J. Herbert. An Augmented Capability Architecture toSupport Lattice Security and Traceability of Access. InProc. 1984 IEEESymposium on Security and Privacy, pages 2–12, Oakland, CA, April 1984.IEEE.
- [KK02] Alan H. Karp and Vana Kalogeraki. The Client Utility as a Peer-to-Peer System. In Revised Papers from the NETWORKING 2002 Workshops on Web Engineering and Peer-to-Peer Computing, pages 260–273, London, UK,2002. Springer-Verlag.
- [KL86] Richard Y. Kain and Carl E. Landwehr. On Access Checking in CapabilitySystems. In Proc. 1986 IEEE Symposium on Security and Privacy, pages95–100, 1986.
- [KM66] Richard M. Karp and Raymond E. Miller. Properties of a Model for Parallel Computations: Determinacy, Termination, Queueing. SIAM J. Appl. Math., 14(6):1390–1411, November 1966.
- [KM88] Kenneth M. Kahn and Mark S. Miller. Language Design and Open Systems. In Bernardo A. Huberman, editor,Ecology of Computation. Elsevier Science Publishers, North-Holland, 1988.
- [Kri80] Saul A. Kripke. Naming and Necessity. Library of Philosophy and Logic.Blackwell, Oxford, 1980.
- [KTMB87] Kenneth M. Kahn, E. Dean Tribble, Mark S. Miller, and Daniel G. Bo-brow. Vulcan: Logical Concurrent Objects. InResearch Directions in Object-Oriented Programming, pages 75–112. MIT Press, 1987.
- [Lac56] Ludwig Lachmann. Capital and its Structure. Kansas City: Sheed Andrewsand McNeel Inc., 1956.
- [Lam73] Butler W. Lampson. A Note on the Confinement Problem.Communicationsof the ACM, 16(10):613–615, 1973.
- [Lam74] Butler W. Lampson. Protection.Operating Systems Review, 8(1):18–24, January 1974.
- [Lam78] Leslie Lamport. Time, Clocks, and the Ordering of Events in a Distributed System. Communications of the ACM, 21(7):558–565, 1978.
- [Lam98] Leslie Lamport. The Part-time Parliament.ACM Transactions on Computer Systems, 16(2):133–169, 1998.
- [Lan92] Charles R. Landau. The Checkpoint Mechanism in KeyKOS. InProc. Second International Workshop on Object Orientation in Operating Systems, pages 86–91. IEEE, September 1992.
- [Lee06] Edward A. Lee. The Problem with Threads. Technical Report UCB/EECS-2006-1, EECS Department, University of California, Berkeley, January 102006. www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.html
- [Lef03] Glyph Lefkowitz. Generalization of Deferred Execution in Python, 2003. python.org/pycon/papers/deferex/.
- [Lev84] Henry M. Levy. Capability-based Computer Systems. Digital Press, 1984.
- [LG86] B. Liskov and J. Guttag.Abstraction and Specification in Program Develope-ment. MIT Press, Cambridge, Mass., 1986.
- [LN79] Hugh C. Lauer and Roger M. Needham. On the Duality of Operating System Structures. In Proc. 2nd International Symposium on Operating Systems, pages 3–19. IRIA, October 1979. ACM Operating System Review.
- [LS76] Butler W. Lampson and Howard E. Sturgis. Reflections on an Operating System Design. Communications of the ACM, 19(4):251–265, May 1976.
Edit this page
Social Share
Twitter Facebook LinkedIn RedditTips Welcome
Feel free to contact me if you're interested in Bitcoin related content / research!
| Bitcoin | DOGE |
|---|---|
| 1A1DZfw4VgpHCgnMjnmfDnMjddKf8xdYbd | DQKkzfJjqnXUD8Z7C3e84vKzvghPe9dXSa |


CardX
Modern Personal Portfolio / CV / vCard / HTML5 Template
Thank you for purchasing CardX, i wish you all the best with this item!
If you have any questions that are beyond the scope of this documentation file, please feel free to contact us anytime.
I would be very grateful if you could rate my item at this link with 5 stars (★★★★★) and leave your feedback, this is very important for me, Thanks! ❤
General Information
- Template version: 1.0.0
- By: Exill
Template files structure
documentation
- The current folderpsd
- Contains PSD filessource
- Contains Source codeTemplate
- Contains Template filescss
- Contains Stylesheets filesfonts
- Contains Font filesimg
- Contains Imagesjs
- Contains JavaScript filesphp
- Contains PHP files*.html
- Main HTML files
Basic index.html
File Structure
<!doctype html>

Notes
For Template customization, please add your custom styles and scripts to css/custom.css and js/custom.js. This will make it easier for you to install template updates.
General Customizations
Themes
This template has Dark and Light theme. You can switch between themes by editing:
<link id="theme-scheme" rel="stylesheet" href="css/themes/main-THEME_NAME.css">
And

Available theme options
main-dark.css main-light.css
Color schemes
We have created 22 various color options, choose your favorite...
To switch colors, you just have to edit the following line in your HTML file:
<link id="color-scheme" rel="stylesheet" href="css/colors/main-COLOR_NAME.css">
Available color options
main-blue.css main-darkred.css main-red.css
main-blueviolet.css main-green.css main-redpink.css
main-brown.css main-indigo.css main-royalblue.css
main-cyan.css main-moodyblue.css main-tangopink.css
main-dark.css main-nightskyblue.css main-teal.css
main-darkblue.css main-orange.css main-yellowgreen.css
main-darkmagenta.css main-pink.css
main-darkpink.css main-purple.css
Icons
We have used Ionicons pack in this Template, you can easily add any icons by getting its name from Here.
Example

To configure this form, open php/contact.php
and edit the following line:
// Email recipient
$EmailTo =<EMAIL_ADDRESS>If the Contact form is not working after configuring it, there are some possibilities that could block this form from sending you submissions:
- Check your spam folder
- Make sure to enable the PHP mail() function on your server
- Upgrade the PHP version running on your server (Contact your hosting provider if you don't know how to upgrade)
Global Classes
.button-main
- The main button style (White).button-scheme
- An additional class for .button-main to inherit the current color scheme (E.g: blue, red...).single-section
- Gives the section a top and bottom padding.alt-bg
- An additional class for .single-section to change the section's background color to a lighter/darker color.element-cover-bg
- Adds an overlay background to the element.color-scheme
- Current schemecolor:
.background-scheme
- Current schemebackground-color:
.anchor-basic
- Removes the the color from
tags
Bootstrap spacing classes
You can also use a wide range of shorthand responsive margin and padding utility classes from Bootstrap.
Section's title & description example

Get in Touch

Feel free to contact me anytime

Note: The above markup is required to be inside a section that has .single-section class in order to work.
Uploading to your Server
Assuming you already have a Domain name and Hosting, You need to get your FTP details from your Hosting control panel, then use those login details to connect to your server using an FTP client such as FileZilla, then you can start uploading your files.
Note: You can also upload the template using the file manager from your control panel
Due to local security restrictions, some features will not work locally. But everything will work once you upload your files to your server.
Support
We will be happy to help you solve any issues. If you have any issues, questions or ideas, please feel free to contact us anytime.
Once again, thank you so much for purchasing this item!
Credits
Check assets-licenses.md
file for assets credits
Designed and Developed with ❤ by Exill.


Base class for MMKP metaheuristic package. More...
#include <MMKP_MetaHeuristic.h>
Public Member Functions | |
| MMKP_MetaHeuristic (MMKPDataSet dataSet, MetaHeuristic_parameters parameters) | |
| Construct MMKP_MetaHeuristic object. More... | |
| MMKP_MetaHeuristic (MMKPDataSet dataSet) | |
| Construct MMKP_MetaHeuristic object. More... | |
| virtual MMKPSolution | operator() (std::vector< MMKPSolution > initialPopulation)=0 |
| Run heuristic and return the best objective function. More... | |
| std::vector< std::tuple< int, float > > | getConvergenceData () |
| Return convergence data, a vector containing the count of function evaluation and current best objective function. More... | |
| int | getConvergenceGeneration () |
| void | quickSort (std::vector< MMKPSolution > &input, int p, int r) |
| Sort a population of MMKPSolution's using quicksort. More... | |
| virtual MMKPSolution | run (std::vector< MMKPSolution > initialPopulation)=0 |
| Run metaheuristic and return the result, the best objective function. | |
| virtual std::vector< MMKPSolution > | runOneGeneration (std::vector< MMKPSolution > population)=0 |
| Run a single iteration of heuristic. | |
| bool | makeFeasible (MMKPSolution &sol) |
| Wrapper function of both feasibility routines. | |
| bool | makeFeasible (MMKPSolution &sol, int mcFeas, int mdFeas) |
| Wrapper function of both feasiblity routines. More... | |
| bool | makeMultiChoiceFeasFixedSurrogate (MMKPSolution &sol) |
| Make multi-choice feasible based on a surrogate constraint created using all costs of an item. More... | |
| bool | makeMultiChoiceFeasFixed_Rand_Surrogate (MMKPSolution &sol) |
| Make multi-choice feasible based on a surrogate constraint created using all costs in an item. More... | |
| bool | makeMultiChoiceFeasMaxProfit (MMKPSolution &sol) |
| Make multi-choice feasible based on profit alone. More... | |
| bool | makeMultiDimFeasFixedSurrogate (MMKPSolution &sol) |
| Make multi-dim feasible based a surrogate constraint created using all costs in an item. More... | |
| bool | makeMultiDimFeasVariableSurrogate (MMKPSolution &sol) |
| Make multi-dim feasible based a surrogate constraint created using a variable number of item's costs's (depending on which violate the problem's resources. More... | |
| bool | makeMultiDimFeasVarMinSurrogate (MMKPSolution &sol) |
| Make multi-dim feasible based a surrogate constraint created using variable number of item's costs's (depending on which violate the problem's resources. More... | |
| bool | makeMultiDimFeasVarMaximizeProfit (MMKPSolution &sol) |
| Make multi-dim feasible based on maximizing profit. More... | |
Protected Member Functions | |
| void | competitiveUpdateSol (MMKPSolution &sol, MMKPSolution &newSol) |
| void | addToUniqueSolutionSet (MMKPSolution sol) |
| Add to unqiue solution set. More... | |
| int | partition (std::vector< MMKPSolution > &input, int p, int r) |
| Quicksort helper funtion. | |
Protected Attributes | |
| MMKPDataSet | dataSet |
| MetaHeuristic_parameters | parameters |
|
std::vector< std::tuple< int, float > > | convergenceData |
| int | currentFuncEvals |
| int | convergenceIteration |
| int | currentGeneration |
Base class for MMKP metaheuristic package.
MMKP_Metaheuristic provides helper functions common to all metaheuristic's such as making a solution feasible and sorting a population.
| MMKP_MetaHeuristic::MMKP_MetaHeuristic | ( | MMKPDataSet | dataSet, |
| MetaHeuristic_parameters | parameters | ||
| ) |
Construct MMKP_MetaHeuristic object.
Param: parameters can customize the algorithm according to an instance of MetaHeuristic_parameters.
| MMKP_MetaHeuristic::MMKP_MetaHeuristic | ( | MMKPDataSet | dataSet | ) |
Construct MMKP_MetaHeuristic object.
Uses default parameters.
|
protected |
Add to unqiue solution set.
If Solution exists this function does nothing, and if not it will be added to unique solutions list.
| std::vector< std::tuple< int, float > > MMKP_MetaHeuristic::getConvergenceData | ( | ) |
Return convergence data, a vector containing the count of function evaluation and current best objective function.
This can be done after running an instance of metaheuristic, else an empty std::vector will be returned.
| bool MMKP_MetaHeuristic::makeFeasible | ( | MMKPSolution & | sol, |
| int | mcFeas, | ||
| int | mdFeas | ||
| ) |
Wrapper function of both feasiblity routines.
Uses function parameters instead of Parameters struct to dictate which feasibilty algorithm to apply.
| bool MMKP_MetaHeuristic::makeMultiChoiceFeasFixed_Rand_Surrogate | ( | MMKPSolution & | sol | ) |
Make multi-choice feasible based on a surrogate constraint created using all costs in an item.
If one item of a class is selected, continue, else if more than one is selected the item with the highest profit-resource ratio is used. If none are selected then randomly choose an item from that class. Return's true if successful, false otherwise.
| bool MMKP_MetaHeuristic::makeMultiChoiceFeasFixedSurrogate | ( | MMKPSolution & | sol | ) |
Make multi-choice feasible based on a surrogate constraint created using all costs of an item.
If one item of a class is selected, continue, else if more than one is selected the item with the highest profit-resource ratio is used. If none are selected choose the item in that class with the hightest profit-resource ratio. Return's true if successful, false otherwise.
| bool MMKP_MetaHeuristic::makeMultiChoiceFeasMaxProfit | ( | MMKPSolution & | sol | ) |
Make multi-choice feasible based on profit alone.
If one item of a class is selected, continue, else if more than one is selected the item with the highest profit is used. If none are selected choose highest profit. Return's true if successful, false otherwise.
| bool MMKP_MetaHeuristic::makeMultiDimFeasFixedSurrogate | ( | MMKPSolution & | sol | ) |
Make multi-dim feasible based a surrogate constraint created using all costs in an item.
Feasibility is constructed using highest difference of Er%/n. Return's true if successful, false otherwise.
Precondition: there must only be one item of each class selected, correctness is not garenteed otherwise.
| bool MMKP_MetaHeuristic::makeMultiDimFeasVariableSurrogate | ( | MMKPSolution & | sol | ) |
Make multi-dim feasible based a surrogate constraint created using a variable number of item's costs's (depending on which violate the problem's resources.
Feasibility is constructed using highest difference of Er%/n. Return's true if successful, false otherwise.
Precondition: there must only be one item of each class selected, correctness is not garenteed otherwise.
| bool MMKP_MetaHeuristic::makeMultiDimFeasVarMaximizeProfit | ( | MMKPSolution & | sol | ) |
Make multi-dim feasible based on maximizing profit.
This procedure is more expensive then others becuase it attempts to gain feasibility while staying at the highest profit via local search heuristic.
Precondition: there must only be one item of each class selected, correctness is not garenteed otherwise.
| bool MMKP_MetaHeuristic::makeMultiDimFeasVarMinSurrogate | ( | MMKPSolution & | sol | ) |
Make multi-dim feasible based a surrogate constraint created using variable number of item's costs's (depending on which violate the problem's resources.
Feasibility is constructed using lowest difference of Er%/n. Return's true if successful, false otherwise.
Precondition: there must only be one item of each class selected, correctness is not garenteed otherwise.
|
pure virtual |
| void MMKP_MetaHeuristic::quickSort | ( | std::vector< MMKPSolution > & | input, |
| int | p, | ||
| int | r | ||
| ) |
Sort a population of MMKPSolution's using quicksort.
Param p is the starting array element and r is the last. Note that r is not equal to the size of input, but the index of the last element to be sorted (in most cases size-1).


MORTAL KOMBAT "Kicks are for Kids" A film review by Bill @1995 and Bill don't need no stinkin copyright.
Rating: 2.5/4 (Not nearly as bad as STREETFIGHTER.)
Preview: Saw the one hour making of/preview on cable, turns out they shot on location in Thailand. The guy who plays Liu Kang is in/famous for many movies in Hong Kong, and he also helped choreograph a bunch of the fighting.
The guy who plays badass Shang Tsung is REALLY REALLY famous, you'll recognize him easily.
The documentary was interesting, and the walkthroughs of the sets were nice--they put a *lot* of detail into 'em, and then you see'm for 30 seconds of the movie.
The Movie
Christopher Lambert (McCloud from Highlander) plays RAIDEN.
The special moves for sub-zero were really cool! (no pun intended.)
Scorpion's fatality was good!
They got Johnny Cage's speed/blur kick just right!
Sonya was okay (what do you expect for a chick?)
Goro was entirely computer generated as every four armed badass should be, but he wasn't utilized to his full potential.
 Get on with the fighting already!
And when it comes time to fight, some of these guys get wasted too quick....

 Yeah!" and "FINISH HIM!"
To its credit, it kept a fairly quick pace, and the *music* really punched it along! Check out the soundtrack!

The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


7. Dimensionality Reduction¶
7.1. Reducing the number of degrees of freedom, overarching view¶
Many Machine Learning problems involve thousands or even millions of features for each training instance. Not only does this make training extremely slow, it can also make it much harder to find a good solution, as we will see. This problem is often referred to as the curse of dimensionality. Fortunately, in real-world problems, it is often possible to reduce the number of features considerably, turning an intractable problem into a tractable one.
Here we will discuss some of the most popular dimensionality reduction techniques: the principal component analysis (PCA), Kernel PCA, and Locally Linear Embedding (LLE). Furthermore, we will start by looking at some simple preprocessing of the data which allow us to rescale the data.
Principal component analysis and its various variants deal with the problem of fitting a low-dimensional affine subspace to a set of of data points in a high-dimensional space. With its family of methods it is one of the most used tools in data modeling, compression and visualization.
7.2. Preprocessing our data¶
Before we proceed however, we will discuss how to preprocess our data. Till now and in connection with our previous examples we have not met so many cases where we are too sensitive to the scaling of our data. Normally the data may need a rescaling and/or may be sensitive to extreme values. Scaling the data renders our inputs much more suitable for the algorithms we want to employ.
Scikit-Learn has several functions which allow us to rescale the data, normally resulting in much better results in terms of various accuracy scores. The StandardScaler function in Scikit-Learn ensures that for each feature/predictor we study the mean value is zero and the variance is one (every column in the design/feature matrix). This scaling has the drawback that it does not ensure that we have a particular maximum or minimum in our data set. Another function included in Scikit-Learn is the MinMaxScaler which ensures that all features are exactly between \(0\) and \(1\). The
7.3. More preprocessing¶
The Normalizer scales each data point such that the feature vector has a euclidean length of one. In other words, it projects a data point on the circle (or sphere in the case of higher dimensions) with a radius of 1. This means every data point is scaled by a different number (by the inverse of it’s length). This normalization is often used when only the direction (or angle) of the data matters, not the length of the feature vector.
The RobustScaler works similarly to the StandardScaler in that it ensures statistical properties for each feature that guarantee that they are on the same scale. However, the RobustScaler uses the median and quartiles, instead of mean and variance. This makes the RobustScaler ignore data points that are very different from the rest (like measurement errors). These odd data points are also called outliers, and might often lead to trouble for other scaling techniques.
7.4. Simple preprocessing examples, Franke function and regression¶
%matplotlib inline
# Common imports
import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import sklearn.linear_model as skl
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler, StandardScaler, Normalizer
from sklearn.svm import SVR
# Where to save the figures and data files
PROJECT_ROOT_DIR = "Results"
FIGURE_ID = "Results/FigureFiles"
DATA_ID = "DataFiles/"
if not os.path.exists(PROJECT_ROOT_DIR):
os.mkdir(PROJECT_ROOT_DIR)
if not os.path.exists(FIGURE_ID):
os.makedirs(FIGURE_ID)
if not os.path.exists(DATA_ID):
os.makedirs(DATA_ID)
def image_path(fig_id):
return os.path.join(FIGURE_ID, fig_id)
def data_path(dat_id):
return os.path.join(DATA_ID, dat_id)
def save_fig(fig_id):
plt.savefig(image_path(fig_id) + ".png", format='png')
def FrankeFunction(x,y):
term1 = 0.75*np.exp(-(0.25*(9*x-2)**2) - 0.25*((9*y-2)**2))
term2 = 0.75*np.exp(-((9*x+1)**2)/49.0 - 0.1*(9*y+1))
term3 = 0.5*np.exp(-(9*x-7)**2/4.0 - 0.25*((9*y-3)**2))
term4 = -0.2*np.exp(-(9*x-4)**2 - (9*y-7)**2)
return term1 + term2 + term3 + term4
def create_X(x, y, n ):
if len(x.shape) > 1:
x = np.ravel(x)
y = np.ravel(y)
N = len(x)
l = int((n+1)*(n+2)/2) # Number of elements in beta
X = np.ones((N,l))
for i in range(1,n+1):
q = int((i)*(i+1)/2)
for k in range(i+1):
X[:,q+k] = (x**(i-k))*(y**k)
return X
# Making meshgrid of datapoints and compute Franke's function
n = 5
N = 1000
x = np.sort(np.random.uniform(0, 1, N))
y = np.sort(np.random.uniform(0, 1, N))
z = FrankeFunction(x, y)
X = create_X(x, y, n=n)
# split in training and test data
X_train, X_test, y_train, y_test = train_test_split(X,z,test_size=0.2)
svm = SVR(gamma='auto',C=10.0)
svm.fit(X_train, y_train)
# The mean squared error and R2 score
print("MSE before scaling: {:.2f}".format(mean_squared_error(svm.predict(X_test), y_test)))
print("R2 score before scaling {:.2f}".format(svm.score(X_test,y_test)))
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
print("Feature min values before scaling:\n {}".format(X_train.min(axis=0)))
print("Feature max values before scaling:\n {}".format(X_train.max(axis=0)))
print("Feature min values after scaling:\n {}".format(X_train_scaled.min(axis=0)))
print("Feature max values after scaling:\n {}".format(X_train_scaled.max(axis=0)))
svm = SVR(gamma='auto',C=10.0)
svm.fit(X_train_scaled, y_train)
print("MSE after scaling: {:.2f}".format(mean_squared_error(svm.predict(X_test_scaled), y_test)))
print("R2 score for scaled data: {:.2f}".format(svm.score(X_test_scaled,y_test)))
MSE before scaling: 0.01
R2 score before scaling 0.94
Feature min values before scaling:
[1.00000000e+00 3.86972479e-03 3.66528972e-03 1.49747700e-05
1.41836625e-05 1.34343487e-05 5.79482386e-08 5.48868704e-08
5.19872323e-08 4.92407803e-08 2.24243736e-10 2.12397083e-10
2.01176282e-10 1.90548268e-10 1.80481726e-10 8.67761543e-13
8.21918258e-13 7.78496845e-13 7.37369358e-13 6.98414609e-13
6.61517814e-13]
Feature max values before scaling:
[1. 0.99791107 0.99418827 0.99582651 0.99211148 0.98841032
0.9937463 0.99003903 0.9863456 0.98266595 0.99167043 0.98797091
0.9842852 0.98061323 0.97695496 0.9895989 0.98590711 0.98222909
0.9785648 0.97491417 0.97127716]
Feature min values after scaling:
[ 0. -1.66745992 -1.72823447 -1.09228638 -1.1114811 -1.1313648
-0.86506245 -0.87590574 -0.88698187 -0.89829265 -0.73687981 -0.74440021
-0.75205856 -0.75985475 -0.76778823 -0.65190491 -0.65755533 -0.66330693
-0.66916042 -0.67511627 -0.68117476]
Feature max values after scaling:
[0. 1.76330702 1.70952132 2.29506297 2.25330559 2.21033632
2.73427311 2.69911557 2.66307432 2.6261167 3.11835914 3.08796041
3.05688927 3.02512212 2.99263434 3.46490261 3.4379033 3.41038453
3.38232999 3.35372259 3.32454443]
MSE after scaling: 0.00
R2 score for scaled data: 0.97
7.5. Simple preprocessing examples, breast cancer data and classification, Support Vector Machines¶
We show here how we can use a simple regression case on the breast cancer data using support vector machines (SVM) as algorithm for classification.
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
from sklearn.svm import SVC
cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
print(X_train.shape)
print(X_test.shape)
svm = SVC(C=100)
svm.fit(X_train, y_train)
print("Test set accuracy: {:.2f}".format(svm.score(X_test,y_test)))
from sklearn.preprocessing import MinMaxScaler, StandardScaler
scaler = MinMaxScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
print("Feature min values before scaling:\n {}".format(X_train.min(axis=0)))
print("Feature max values before scaling:\n {}".format(X_train.max(axis=0)))
print("Feature min values before scaling:\n {}".format(X_train_scaled.min(axis=0)))
print("Feature max values before scaling:\n {}".format(X_train_scaled.max(axis=0)))
svm.fit(X_train_scaled, y_train)
print("Test set accuracy scaled data with Min-Max scaling: {:.2f}".format(svm.score(X_test_scaled,y_test)))
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
svm.fit(X_train_scaled, y_train)
print("Test set accuracy scaled data with Standar Scaler: {:.2f}".format(svm.score(X_test_scaled,y_test)))
(426, 30)
(143, 30)
Test set accuracy: 0.94
Feature min values before scaling:
[6.981e+00 9.710e+00 4.379e+01 1.435e+02 5.263e-02 1.938e-02 0.000e+00
0.000e+00 1.060e-01 4.996e-02 1.115e-01 3.628e-01 7.570e-01 7.228e+00
1.713e-03 2.252e-03 0.000e+00 0.000e+00 7.882e-03 8.948e-04 7.930e+00
1.202e+01 5.041e+01 1.852e+02 7.117e-02 2.729e-02 0.000e+00 0.000e+00
1.565e-01 5.504e-02]
Feature max values before scaling:
[2.811e+01 3.381e+01 1.885e+02 2.501e+03 1.447e-01 3.114e-01 4.268e-01
2.012e-01 3.040e-01 9.744e-02 2.873e+00 4.885e+00 2.198e+01 5.422e+02
2.333e-02 1.064e-01 3.960e-01 5.279e-02 6.146e-02 2.984e-02 3.604e+01
4.954e+01 2.512e+02 4.254e+03 2.226e-01 1.058e+00 1.252e+00 2.903e-01
6.638e-01 2.075e-01]
Feature min values before scaling:
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.]
Feature max values before scaling:
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
1. 1. 1. 1. 1. 1.]
Test set accuracy scaled data with Min-Max scaling: 0.97
Test set accuracy scaled data with Standar Scaler: 0.96
7.6. More on Cancer Data, now with Logistic Regression¶
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
cancer = load_breast_cancer()
# Set up training data
X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
logreg = LogisticRegression()
logreg.fit(X_train, y_train)
print("Test set accuracy: {:.2f}".format(logreg.score(X_test,y_test)))
# Scale data
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
logreg.fit(X_train_scaled, y_train)
print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
Test set accuracy: 0.95
Test set accuracy scaled data: 0.96
/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:762: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
7.7. Why should we think of reducing the dimensionality¶
In addition to the plot of the features, we study now also the covariance (and the correlation matrix). We use also Pandas to compute the correlation matrix.
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
cancer = load_breast_cancer()
import pandas as pd
# Making a data frame
cancerpd = pd.DataFrame(cancer.data, columns=cancer.feature_names)
fig, axes = plt.subplots(15,2,figsize=(10,20))
malignant = cancer.data[cancer.target == 0]
benign = cancer.data[cancer.target == 1]
ax = axes.ravel()
for i in range(30):
_, bins = np.histogram(cancer.data[:,i], bins =50)
ax[i].hist(malignant[:,i], bins = bins, alpha = 0.5)
ax[i].hist(benign[:,i], bins = bins, alpha = 0.5)
ax[i].set_title(cancer.feature_names[i])
ax[i].set_yticks(())
ax[0].set_xlabel("Feature magnitude")
ax[0].set_ylabel("Frequency")
ax[0].legend(["Malignant", "Benign"], loc ="best")
fig.tight_layout()
plt.show()
import seaborn as sns
correlation_matrix = cancerpd.corr().round(1)
# use the heatmap function from seaborn to plot the correlation matrix
# annot = True to print the values inside the square
sns.heatmap(data=correlation_matrix, annot=True)
plt.show()
#print eigvalues of correlation matrix
EigValues, EigVectors = np.linalg.eig(correlation_matrix)
print(EigValues)
[ 1.33026664e+01 5.69238112e+00 2.83259341e+00 1.99126621e+00
1.69095941e+00 1.19614372e+00 7.13037918e-01 5.77804918e-01
5.14549644e-01 4.30579019e-01 3.58977964e-01 -1.91269450e-01
2.86195387e-01 -1.59310961e-01 -1.38483045e-01 2.44786221e-01
-8.79318674e-02 1.91811329e-01 1.70896266e-01 -7.17683028e-02
1.39860289e-01 1.18721828e-01 -4.83194721e-02 8.92983892e-02
6.81402316e-02 6.01968016e-02 3.31819744e-02 1.56432255e-02
-1.61028757e-02 -6.50565973e-03]
In the above example we note two things. In the first plot we display the overlap of benign and malignant tumors as functions of the various features in the Wisconsing breast cancer data set. We see that for some of the features we can distinguish clearly the benign and malignant cases while for other features we cannot. This can point to us which features may be of greater interest when we wish to classify a benign or not benign tumour.
In the second figure we have computed the so-called correlation matrix, which in our case with thirty features becomes a \(30\times 30\) matrix.
We constructed this matrix using pandas via the statements
cancerpd = pd.DataFrame(cancer.data, columns=cancer.feature_names)
and then
correlation_matrix = cancerpd.corr().round(1)
Diagonalizing this matrix we can in turn say something about which features are of relevance and which are not. But before we proceed we need to define covariance and correlation matrices. This leads us to the classical Principal Component Analysis (PCA) theorem with applications.
7.8. Basic ideas of the Principal Component Analysis (PCA)¶
The principal component analysis deals with the problem of fitting a low-dimensional affine subspace \(S\) of dimension \(d\) much smaller than the totaldimension \(D\) of the problem at hand (our data set). Mathematically it can be formulated as a statistical problem or a geometric problem. In our discussion of the theorem for the classical PCA, we will stay with a statistical approach. This is also what set the scene historically which for the PCA.
We have a data set defined by a design/feature matrix \(\boldsymbol{X}\) (see below for its definition)
Each data point is determined by \(p\) extrinsic (measurement) variables
We may want to ask the following question: Are there fewer intrinsic variables (say \(d << p\)) that still approximately describe the data?
If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do.
7.9. Introducing the Covariance and Correlation functions¶
Before we discuss the PCA theorem, we need to remind ourselves about the definition of the covariance and the correlation function. These are quantities
Suppose we have defined two vectors \(\hat{x}\) and \(\hat{y}\) with \(n\) elements each. The covariance matrix \(\boldsymbol{C}\) is defined as
where for example
With this definition and recalling that the variance is defined as
we can rewrite the covariance matrix as
The covariance takes values between zero and infinity and may thus lead to problems with loss of numerical precision for particularly large values. It is common to scale the covariance matrix by introducing instead the correlation matrix defined via the so-called correlation function
The correlation function is then given by values \(\mathrm{corr}[\boldsymbol{x},\boldsymbol{y}] \in [-1,1]\). This avoids eventual problems with too large values. We can then define the correlation matrix for the two vectors \(\boldsymbol{x}\) and \(\boldsymbol{y}\) as
In the above example this is the function we constructed using pandas.
7.10. Correlation Function and Design/Feature Matrix¶
In our derivation of the various regression algorithms like Ordinary Least Squares or Ridge regression we defined the design/feature matrix \(\boldsymbol{X}\) as
with \(\boldsymbol{X}\in {\mathbb{R}}^{n\times p}\), with the predictors/features \(p\) refering to the column numbers and the entries \(n\) being the row elements. We can rewrite the design/feature matrix in terms of its column vectors as
with a given vector
With these definitions, we can now rewrite our \(2\times 2\) correaltion/covariance matrix in terms of a moe general design/feature matrix \(\boldsymbol{X}\in {\mathbb{R}}^{n\times p}\). This leads to a \(p\times p\) covariance matrix for the vectors \(\boldsymbol{x}_i\) with \(i=0,1,\dots,p-1\)
and the correlation matrix
7.11. Covariance Matrix Examples¶
The Numpy function np.cov calculates the covariance elements using the factor \(1/(n-1)\) instead of \(1/n\) since it assumes we do not have the exact mean values. The following simple function uses the np.vstack function which takes each vector of dimension \(1\times n\) and produces a \(2\times n\) matrix \(\boldsymbol{W}\)
which in turn is converted into into the \(2\times 2\) covariance matrix \(\boldsymbol{C}\) via the Numpy function np.cov(). We note that we can also calculate the mean value of each set of samples \(\boldsymbol{x}\) etc using the Numpy function np.mean(x). We can also extract the eigenvalues of the covariance matrix through the np.linalg.eig() function.
# Importing various packages
import numpy as np
n = 100
x = np.random.normal(size=n)
print(np.mean(x))
y = 4+3*x+np.random.normal(size=n)
print(np.mean(y))
W = np.vstack((x, y))
C = np.cov(W)
print(C)
0.11455861677725063
4.414311328416008
[[1.04460586 2.98508511]
[2.98508511 9.60711152]]
7.12. Correlation Matrix¶
The previous example can be converted into the correlation matrix by simply scaling the matrix elements with the variances. We should also subtract the mean values for each column. This leads to the following code which sets up the correlations matrix for the previous example in a more brute force way. Here we scale the mean values for each column of the design matrix, calculate the relevant mean values and variances and then finally set up the \(2\times 2\) correlation matrix (since we have only two vectors).
import numpy as np
n = 100
# define two vectors
x = np.random.random(size=n)
y = 4+3*x+np.random.normal(size=n)
#scaling the x and y vectors
x = x - np.mean(x)
y = y - np.mean(y)
variance_x = np.sum(x@x)/n
variance_y = np.sum(y@y)/n
print(variance_x)
print(variance_y)
cov_xy = np.sum(x@y)/n
cov_xx = np.sum(x@x)/n
cov_yy = np.sum(y@y)/n
C = np.zeros((2,2))
C[0,0]= cov_xx/variance_x
C[1,1]= cov_yy/variance_y
C[0,1]= cov_xy/np.sqrt(variance_y*variance_x)
C[1,0]= C[0,1]
print(C)
0.07650519724946082
1.5077307406836722
[[1. 0.63582252]
[0.63582252 1. ]]
We see that the matrix elements along the diagonal are one as they should be and that the matrix is symmetric. Furthermore, diagonalizing this matrix we easily see that it is a positive definite matrix.
The above procedure with numpy can be made more compact if we use pandas.
7.13. Correlation Matrix with Pandas¶
We whow here how we can set up the correlation matrix using pandas, as done in this simple code
import numpy as np
import pandas as pd
n = 10
x = np.random.normal(size=n)
x = x - np.mean(x)
y = 4+3*x+np.random.normal(size=n)
y = y - np.mean(y)
X = (np.vstack((x, y))).T
print(X)
Xpd = pd.DataFrame(X)
print(Xpd)
correlation_matrix = Xpd.corr()
print(correlation_matrix)
[[-0.21054401 -1.67179046]
[-0.27924545 -1.98111498]
[-1.24089109 -3.51965915]
[ 2.17042981 6.77708985]
[-0.36489938 -0.63821317]
[-1.61222872 -5.67668711]
[-0.67184873 -0.62798419]
[ 1.21860807 4.36876959]
[ 0.0351748 -0.93789716]
[ 0.95544469 3.90748678]]
0 1
0 -0.210544 -1.671790
1 -0.279245 -1.981115
2 -1.240891 -3.519659
3 2.170430 6.777090
4 -0.364899 -0.638213
5 -1.612229 -5.676687
6 -0.671849 -0.627984
7 1.218608 4.368770
8 0.035175 -0.937897
9 0.955445 3.907487
0 1
0 1.000000 0.972249
1 0.972249 1.000000
We expand this model to the Franke function discussed above.
7.14. Correlation Matrix with Pandas and the Franke function¶
# Common imports
import numpy as np
import pandas as pd
def FrankeFunction(x,y):
term1 = 0.75*np.exp(-(0.25*(9*x-2)**2) - 0.25*((9*y-2)**2))
term2 = 0.75*np.exp(-((9*x+1)**2)/49.0 - 0.1*(9*y+1))
term3 = 0.5*np.exp(-(9*x-7)**2/4.0 - 0.25*((9*y-3)**2))
term4 = -0.2*np.exp(-(9*x-4)**2 - (9*y-7)**2)
return term1 + term2 + term3 + term4
def create_X(x, y, n ):
if len(x.shape) > 1:
x = np.ravel(x)
y = np.ravel(y)
N = len(x)
l = int((n+1)*(n+2)/2) # Number of elements in beta
X = np.ones((N,l))
for i in range(1,n+1):
q = int((i)*(i+1)/2)
for k in range(i+1):
X[:,q+k] = (x**(i-k))*(y**k)
return X
# Making meshgrid of datapoints and compute Franke's function
n = 4
N = 100
x = np.sort(np.random.uniform(0, 1, N))
y = np.sort(np.random.uniform(0, 1, N))
z = FrankeFunction(x, y)
X = create_X(x, y, n=n)
Xpd = pd.DataFrame(X)
# subtract the mean values and set up the covariance matrix
Xpd = Xpd - Xpd.mean()
covariance_matrix = Xpd.cov()
print(covariance_matrix)
0 1 2 3 4 5 6 7 \
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.0 0.062851 0.073533 0.065769 0.068538 0.069947 0.059538 0.060320
2 0.0 0.073533 0.087625 0.078561 0.082559 0.084744 0.071705 0.072983
3 0.0 0.065769 0.078561 0.072769 0.076696 0.079022 0.068407 0.069803
4 0.0 0.068538 0.082559 0.076696 0.081211 0.083977 0.072511 0.074206
5 0.0 0.069947 0.084744 0.079022 0.083977 0.087102 0.075127 0.077070
6 0.0 0.059538 0.071705 0.068407 0.072511 0.075127 0.066156 0.067782
7 0.0 0.060320 0.072983 0.069803 0.074206 0.077070 0.067782 0.069583
8 0.0 0.060735 0.073737 0.070715 0.075353 0.078424 0.068928 0.070878
9 0.0 0.060967 0.074217 0.071359 0.076188 0.079434 0.069792 0.071870
10 0.0 0.052371 0.063245 0.061909 0.065807 0.068404 0.061247 0.062897
11 0.0 0.052613 0.063714 0.062491 0.066554 0.069297 0.062003 0.063759
12 0.0 0.052756 0.064027 0.062920 0.067118 0.069987 0.062592 0.064441
13 0.0 0.052853 0.064260 0.063261 0.067574 0.070552 0.063079 0.065009
14 0.0 0.052933 0.064453 0.063553 0.067966 0.071040 0.063502 0.065504
8 9 10 11 12 13 14
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.060735 0.060967 0.052371 0.052613 0.052756 0.052853 0.052933
2 0.073737 0.074217 0.063245 0.063714 0.064027 0.064260 0.064453
3 0.070715 0.071359 0.061909 0.062491 0.062920 0.063261 0.063553
4 0.075353 0.076188 0.065807 0.066554 0.067118 0.067574 0.067966
5 0.078424 0.079434 0.068404 0.069297 0.069987 0.070552 0.071040
6 0.068928 0.069792 0.061247 0.062003 0.062592 0.063079 0.063502
7 0.070878 0.071870 0.062897 0.063759 0.064441 0.065009 0.065504
8 0.072303 0.073408 0.064109 0.065065 0.065830 0.066473 0.067035
9 0.073408 0.074615 0.065054 0.066094 0.066934 0.067645 0.068268
10 0.064109 0.065054 0.057780 0.058594 0.059248 0.059799 0.060281
11 0.065065 0.066094 0.058594 0.059478 0.060193 0.060800 0.061333
12 0.065830 0.066934 0.059248 0.060193 0.060965 0.061622 0.062202
13 0.066473 0.067645 0.059799 0.060800 0.061622 0.062326 0.062949
14 0.067035 0.068268 0.060281 0.061333 0.062202 0.062949 0.063612
We note here that the covariance is zero for the first rows and columns since all matrix elements in the design matrix were set to one (we are fitting the function in terms of a polynomial of degree \(n\)).
This means that the variance for these elements will be zero and will cause problems when we set up the correlation matrix. We can simply drop these elements and construct a correlation matrix without these elements.
7.15. Rewriting the Covariance and/or Correlation Matrix¶
We can rewrite the covariance matrix in a more compact form in terms of the design/feature matrix \(\boldsymbol{X}\) as
To see this let us simply look at a design matrix \(\boldsymbol{X}\in {\mathbb{R}}^{2\times 2}\)
If we then compute the expectation value
which is just
where we wrote $\(\boldsymbol{C}[\boldsymbol{x}_0,\boldsymbol{x}_1] = \boldsymbol{C}[\boldsymbol{x}]\)\( to indicate that this the covariance of the vectors \)\boldsymbol{x}\( of the design/feature matrix \)\boldsymbol{X}$.
It is easy to generalize this to a matrix \(\boldsymbol{X}\in {\mathbb{R}}^{n\times p}\).
7.16. Towards the PCA theorem¶
We have that the covariance matrix (the correlation matrix involves a simple rescaling) is given as
Let us now assume that we can perform a series of orthogonal transformations where we employ some orthogonal matrices \(\boldsymbol{S}\). These matrices are defined as \(\boldsymbol{S}\in {\mathbb{R}}^{p\times p}\) and obey the orthogonality requirements \(\boldsymbol{S}\boldsymbol{S}^T=\boldsymbol{S}^T\boldsymbol{S}=\boldsymbol{I}\). The matrix can be written out in terms of the column vectors \(\boldsymbol{s}_i\) as \(\boldsymbol{S}=[\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}]\) and \(\boldsymbol{s}_i \in {\mathbb{R}}^{p}\).
Assume also that there is a transformation \(\boldsymbol{S}\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T=\boldsymbol{C}[\boldsymbol{y}]\) such that the new matrix \(\boldsymbol{C}[\boldsymbol{y}]\) is diagonal with elements \([\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}]\).
That is we have
since the matrix \(\boldsymbol{S}\) is not a data dependent matrix. Multiplying with \(\boldsymbol{S}^T\) from the left we have
and since \(\boldsymbol{C}[\boldsymbol{y}]\) is diagonal we have for a given eigenvalue \(i\) of the covariance matrix that
In the derivation of the PCA theorem we will assume that the eigenvalues are ordered in descending order, that is \(\lambda_0 > \lambda_1 > \dots > \lambda_{p-1}\).
The eigenvalues tell us then how much we need to stretch the corresponding eigenvectors. Dimensions with large eigenvalues have thus large variations (large variance) and define therefore useful dimensions. The data points are more spread out in the direction of these eigenvectors. Smaller eigenvalues mean on the other hand that the corresponding eigenvectors are shrunk accordingly and the data points are tightly bunched together and there is not much variation in these specific directions. Hopefully then we could leave it out dimensions where the eigenvalues are very small. If \(p\) is very large, we could then aim at reducing \(p\) to \(l << p\) and handle only \(l\) features/predictors.
7.17. The Algorithm before theorem¶
Here’s how we would proceed in setting up the algorithm for the PCA, see also discussion below here.
Set up the datapoints for the design/feature matrix \(\boldsymbol{X}\) with \(\boldsymbol{X}\in {\mathbb{R}}^{n\times p}\), with the predictors/features \(p\) referring to the column numbers and the entries \(n\) being the row elements.
Center the data by subtracting the mean value for each column. This leads to a new matrix \(\boldsymbol{X}\rightarrow \overline{\boldsymbol{X}}\).
Compute then the covariance/correlation matrix \(\mathbb{E}[\overline{\boldsymbol{X}}\overline{\boldsymbol{X}}^T]\).
Find the eigenpairs of \(\boldsymbol{C}\) with eigenvalues \([\lambda_0,\lambda_1,\dots,\lambda_{p-1}]\) and eigenvectors \([\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}]\).
Order the eigenvalue (and the eigenvectors accordingly) in order of decreasing eigenvalues.
Keep only those \(l\) eigenvalues larger than a selected threshold value, discarding thus \(p-l\) features since we expect small variations in the data here.
7.18. Writing our own PCA code¶
We will use a simple example first with two-dimensional data drawn from a multivariate normal distribution with the following mean and covariance matrix:
Note that the mean refers to each column of data. We will generate \(n = 1000\) points \(X = \{ x_1, \ldots, x_N \}\) from this distribution, and store them in the \(1000 \times 2\) matrix \(\boldsymbol{X}\).
The following Python code aids in setting up the data and writing out the design matrix. Note that the function multivariate returns also the covariance discussed above and that it is defined by dividing by \(n-1\) instead of \(n\).
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.display import display
n = 10000
mean = (-1, 2)
cov = [[4, 2], [2, 2]]
X = np.random.multivariate_normal(mean, cov, n)
Now we are going to implement the PCA algorithm. We will break it down into various substeps.
7.18.1. Compute the sample mean and center the data¶
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is
and the mean-centered data \(\bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_n \}\) takes the form
When you are done with these steps, print out \(\mu_n\) to verify it is close to \(\mu\) and plot your mean centered data to verify it is centered at the origin! Compare your code with the functionality from Scikit-Learn discussed above. The following code elements perform these operations using pandas or using our own functionality for doing so. The latter, using numpy is rather simple through the mean() function.
df = pd.DataFrame(X)
# Pandas does the centering for us
df = df -df.mean()
# we center it ourselves
X_centered = X - X.mean(axis=0)
Alternatively, we could use the functions we discussed earlier for scaling the data set. That is, we could have used the StandardScaler function in Scikit-Learn, a function which ensures that for each feature/predictor we study the mean value is zero and the variance is one (every column in the design/feature matrix). You would then not get the same results, since we divide by the variance. The diagonal covariance matrix elements will then be one, while the non-diagonal ones need to be divided by \(2\sqrt{2}\) for our specific case.
7.18.2. Compute the sample covariance¶
Now we are going to use the mean centered data to compute the sample covariance of the data by using the following equation
where the data points \(x_i \in \mathbb{R}^p\) (here in this example \(p = 2\)) are column vectors and \(x^T\) is the transpose of \(x\). We can write our own code or simply use either the functionaly of numpy or that of pandas, as follows
print(df.cov())
print(np.cov(X_centered.T))
0 1
0 3.963873 1.979586
1 1.979586 1.976796
[[3.96387325 1.97958566]
[1.97958566 1.97679551]]
Note that the way we define the covariance matrix here has a factor \(n-1\) instead of \(n\). This is included in the cov() function by numpy and pandas. Our own code here is not very elegant and asks for obvious improvements. It is tailored to this specific \(2\times 2\) covariance matrix.
# extract the relevant columns from the centered design matrix of dim n x 2
x = X_centered[:,0]
y = X_centered[:,1]
Cov = np.zeros((2,2))
Cov[0,1] = np.sum(x.T@y)/(n-1.0)
Cov[0,0] = np.sum(x.T@x)/(n-1.0)
Cov[1,1] = np.sum(y.T@y)/(n-1.0)
Cov[1,0]= Cov[0,1]
print("Centered covariance using own code")
print(Cov)
plt.plot(x, y, 'x')
plt.axis('equal')
plt.show()
Centered covariance using own code
[[3.96387325 1.97958566]
[1.97958566 1.97679551]]
Depending on the number of points \(n\), we will get results that are close to the covariance values defined above. The plot shows how the data are clustered around a line with slope close to one. Is this expected?
7.18.3. Diagonalize the sample covariance matrix to obtain the principal components¶
Now we are ready to solve for the principal components! To do so we diagonalize the sample covariance matrix \(\Sigma\). We can use the function np.linalg.eig to do so. It will return the eigenvalues and eigenvectors of \(\Sigma\). Once we have these we can perform the following tasks:
We compute the percentage of the total variance captured by the first principal component
We plot the mean centered data and lines along the first and second principal components
Then we project the mean centered data onto the first and second principal components, and plot the projected data.
Finally, we approximate the data as
where \(v_0\) is the first principal component.
Collecting all these steps we can write our own PCA function and compare this with the functionality included in Scikit-Learn.
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this in order to address the above questions.
# diagonalize and obtain eigenvalues, not necessarily sorted
EigValues, EigVectors = np.linalg.eig(Cov)
# sort eigenvectors and eigenvalues
#permute = EigValues.argsort()
#EigValues = EigValues[permute]
#EigVectors = EigVectors[:,permute]
print("Eigenvalues of Covariance matrix")
for i in range(2):
print(EigValues[i])
FirstEigvector = EigVectors[:,0]
SecondEigvector = EigVectors[:,1]
print("First eigenvector")
print(FirstEigvector)
print("Second eigenvector")
print(SecondEigvector)
#thereafter we do a PCA with Scikit-learn
from sklearn.decomposition import PCA
pca = PCA(n_components = 2)
X2Dsl = pca.fit_transform(X)
print("Eigenvector of largest eigenvalue")
print(pca.components_.T[:, 0])
Eigenvalues of Covariance matrix
5.185256249406441
0.7554125101643117
First eigenvector
[0.85104821 0.52508756]
Second eigenvector
[-0.52508756 0.85104821]
Eigenvector of largest eigenvalue
[-0.85104821 -0.52508756]
This code does not contain all the above elements, but it shows how we can use Scikit-Learn to extract the eigenvector which corresponds to the largest eigenvalue. Try to address the questions we pose before the above code. Try also to change the values of the covariance matrix by making one of the diagonal elements much larger than the other. What do you observe then?
7.19. Classical PCA Theorem¶
We assume now that we have a design matrix \(\boldsymbol{X}\) which has been centered as discussed above. For the sake of simplicity we skip the overline symbol. The matrix is defined in terms of the various column vectors \([\boldsymbol{x}_0,\boldsymbol{x}_1,\dots, \boldsymbol{x}_{p-1}]\) each with dimension \(\boldsymbol{x}\in {\mathbb{R}}^{n}\).
We assume also that we have an orthogonal transformation \(\boldsymbol{W}\in {\mathbb{R}}^{p\times p}\). We define the reconstruction error (which is similar to the mean squared error we have seen before) as
with \(\overline{\boldsymbol{x}}_i = \boldsymbol{W}\boldsymbol{z}_i\), where \(\boldsymbol{z}_i\) is a row vector with dimension \({\mathbb{R}}^{n}\) of the matrix \(\boldsymbol{Z}\in{\mathbb{R}}^{p\times n}\). When doing PCA we want to reduce this dimensionality.
The PCA theorem states that minimizing the above reconstruction error corresponds to setting \(\boldsymbol{W}=\boldsymbol{S}\), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \(\boldsymbol{z}_i\) with at most \(l\) vectors, with \(l << p\), defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix).
The proof which follows will be updated by mid January 2020.
7.20. Proof of the PCA Theorem¶
To show the PCA theorem let us start with the assumption that there is one vector \(\boldsymbol{w}_0\) which corresponds to a solution which minimized the reconstruction error \(J\). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \(\boldsymbol{w}_0\) and \(\boldsymbol{z}_0\) as
which we can rewrite due to the orthogonality of \(\boldsymbol{w}_i\) as
Minimizing \(J\) with respect to the unknown parameters \(z_{0i}\) we obtain that
where the vectors on the rhs are known.
7.21. PCA Proof continued¶
We have now found the unknown parameters \(z_{i0}\). These correspond to the projected coordinates and we can write
We can show that the variance of the projected coordinates defined by \(\boldsymbol{w}_0^T\boldsymbol{x}_i\) are given by
since the expectation value of
where we have used the fact that our data are centered.
Recalling our definition of the covariance as
we have thus that
We are almost there, we have obtained a relation between minimizing the reconstruction error and the variance and the covariance matrix. Minimizing the error is equivalent to maximizing the variance of the projected data.
7.22. The final step¶
We could trivially maximize the variance of the projection (and thereby minimize the error in the reconstruction function) by letting the norm-2 of \(\boldsymbol{w}_0\) go to infinity. However, this norm since we want the matrix \(\boldsymbol{W}\) to be an orthogonal matrix, is constrained by \(\vert\vert \boldsymbol{w}_0 \vert\vert_2^2=1\). Imposing this condition via a Lagrange multiplier we can then in turn maximize
Taking the derivative with respect to \(\boldsymbol{w}_0\) we obtain
meaning that
The direction that maximizes the variance (or minimizes the construction error) is an eigenvector of the covariance matrix! If we left multiply with \(\boldsymbol{w}_0^T\) we have the variance of the projected data is
If we want to maximize the variance (minimize the construction error) we simply pick the eigenvector of the covariance matrix with the largest eigenvalue. This establishes the link between the minimization of the reconstruction function \(J\) in terms of an orthogonal matrix and the maximization of the variance and thereby the covariance of our observations encoded in the design/feature matrix \(\boldsymbol{X}\).
The proof for the other eigenvectors \(\boldsymbol{w}_1,\boldsymbol{w}_2,\dots\) can be established by applying the above arguments and using the fact that our basis of eigenvectors is orthogonal, see Murphy chapter 12.2. The discussion in chapter 12.2 of Murphy’s text has also a nice link with the Singular Value Decomposition theorem. For categorical data, see chapter 12.4 and discussion therein.
Additional part of the proof for the other eigenvectors will be added by mid January 2020.
7.23. Geometric Interpretation and link with Singular Value Decomposition¶
This material will be added by mid January 2020.
7.24. Principal Component Analysis¶
Principal Component Analysis (PCA) is by far the most popular dimensionality reduction algorithm. First it identifies the hyperplane that lies closest to the data, and then it projects the data onto it.
The following Python code uses NumPy’s svd() function to obtain all the principal components of the training set, then extracts the first two principal components. First we center the data using either pandas or our own code
import numpy as np
import pandas as pd
from IPython.display import display
np.random.seed(100)
# setting up a 10 x 5 vanilla matrix
rows = 10
cols = 5
X = np.random.randn(rows,cols)
df = pd.DataFrame(X)
# Pandas does the centering for us
df = df -df.mean()
display(df)
# we center it ourselves
X_centered = X - X.mean(axis=0)
# Then check the difference between pandas and our own set up
print(X_centered-df)
#Now we do an SVD
U, s, V = np.linalg.svd(X_centered)
c1 = V.T[:, 0]
c2 = V.T[:, 1]
W2 = V.T[:, :2]
X2D = X_centered.dot(W2)
print(X2D)
| 0 | 1 | 2 | 3 | 4 | |
|---|---|---|---|---|---|
| 0 | -1.574465 | 0.259153 | 1.197370 | 0.147400 | 0.649382 |
| 1 | 0.689519 | 0.137652 | -1.025709 | 0.210340 | -0.076938 |
| 2 | -0.282727 | 0.351636 | -0.539261 | 1.216683 | 0.340782 |
| 3 | 0.070889 | -0.614808 | 1.074067 | -0.038300 | -1.450257 |
| 4 | 1.794282 | 1.458078 | -0.207545 | -0.442600 | -0.147420 |
| 5 | 1.112383 | 0.647473 | 1.405890 | 0.073598 | -0.276263 |
| 6 | 0.397700 | -1.526744 | -0.712018 | 1.216290 | 0.418506 |
| 7 | -0.280647 | 1.106095 | -1.646283 | -0.956563 | -1.564374 |
| 8 | -0.369139 | -0.751699 | 0.051649 | -0.213103 | 0.967809 |
| 9 | -1.557795 | -1.066837 | 0.401842 | -1.213743 | 1.138775 |
0 1 2 3 4
0 0.0 0.0 0.0 0.0 0.0
1 0.0 0.0 0.0 0.0 0.0
2 0.0 0.0 0.0 0.0 0.0
3 0.0 0.0 0.0 0.0 0.0
4 0.0 0.0 0.0 0.0 0.0
5 0.0 0.0 0.0 0.0 0.0
6 0.0 0.0 0.0 0.0 0.0
7 0.0 0.0 0.0 0.0 0.0
8 0.0 0.0 0.0 0.0 0.0
9 0.0 0.0 0.0 0.0 0.0
[[-1.5378811 -0.94639099]
[ 0.86145244 0.89288636]
[-0.00445655 0.81633628]
[ 0.07145103 -1.00433417]
[ 2.03707133 -0.48476997]
[ 0.72174172 -1.4557763 ]
[-0.55854694 1.60673226]
[ 1.6999536 0.43766686]
[-1.10405456 0.31718909]
[-2.18673098 -0.17953942]]
PCA assumes that the dataset is centered around the origin. Scikit-Learn’s PCA classes take care of centering the data for you. However, if you implement PCA yourself (as in the preceding example), or if you use other libraries, don’t forget to center the data first.
Once you have identified all the principal components, you can reduce the dimensionality of the dataset down to \(d\) dimensions by projecting it onto the hyperplane defined by the first \(d\) principal components. Selecting this hyperplane ensures that the projection will preserve as much variance as possible.
W2 = V.T[:, :2]
X2D = X_centered.dot(W2)
7.25. PCA and scikit-learn¶
Scikit-Learn’s PCA class implements PCA using SVD decomposition just like we did before. The following code applies PCA to reduce the dimensionality of the dataset down to two dimensions (note that it automatically takes care of centering the data):
#thereafter we do a PCA with Scikit-learn
from sklearn.decomposition import PCA
pca = PCA(n_components = 2)
X2D = pca.fit_transform(X)
print(X2D)
[[ 1.5378811 -0.94639099]
[-0.86145244 0.89288636]
[ 0.00445655 0.81633628]
[-0.07145103 -1.00433417]
[-2.03707133 -0.48476997]
[-0.72174172 -1.4557763 ]
[ 0.55854694 1.60673226]
[-1.6999536 0.43766686]
[ 1.10405456 0.31718909]
[ 2.18673098 -0.17953942]]
After fitting the PCA transformer to the dataset, you can access the principal components using the components variable (note that it contains the PCs as horizontal vectors, so, for example, the first principal component is equal to
pca.components_.T[:, 0].
File "<ipython-input-19-17314f270d45>", line 1
pca.components_.T[:, 0].
^
SyntaxError: invalid syntax
Another very useful piece of information is the explained variance ratio of each principal component, available via the \(explained\_variance\_ratio\) variable. It indicates the proportion of the dataset’s variance that lies along the axis of each principal component.
7.26. Back to the Cancer Data¶
We can now repeat the above but applied to real data, in this case our breast cancer data. Here we compute performance scores on the training data using logistic regression.
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
logreg = LogisticRegression()
logreg.fit(X_train, y_train)
print("Train set accuracy from Logistic Regression: {:.2f}".format(logreg.score(X_train,y_train)))
# We scale the data
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
# Then perform again a log reg fit
logreg.fit(X_train_scaled, y_train)
print("Train set accuracy scaled data: {:.2f}".format(logreg.score(X_train_scaled,y_train)))
#thereafter we do a PCA with Scikit-learn
from sklearn.decomposition import PCA
pca = PCA(n_components = 2)
X2D_train = pca.fit_transform(X_train_scaled)
# and finally compute the log reg fit and the score on the training data
logreg.fit(X2D_train,y_train)
print("Train set accuracy scaled and PCA data: {:.2f}".format(logreg.score(X2D_train,y_train)))
We see that our training data after the PCA decomposition has a performance similar to the non-scaled data.
7.27. More on the PCA¶
Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to choose the number of dimensions that add up to a sufficiently large portion of the variance (e.g., 95%). Unless, of course, you are reducing dimensionality for data visualization — in that case you will generally want to reduce the dimensionality down to 2 or 3. The following code computes PCA without reducing dimensionality, then computes the minimum number of dimensions required to preserve 95% of the training set’s variance:
pca = PCA()
pca.fit(X)
cumsum = np.cumsum(pca.explained_variance_ratio_)
d = np.argmax(cumsum >= 0.95) + 1
You could then set \(n\_components=d\) and run PCA again. However, there is a much better option: instead of specifying the number of principal components you want to preserve, you can set \(n\_components\) to be a float between 0.0 and 1.0, indicating the ratio of variance you wish to preserve:
pca = PCA(n_components=0.95)
X_reduced = pca.fit_transform(X)
7.28. Incremental PCA¶
One problem with the preceding implementation of PCA is that it requires the whole training set to fit in memory in order for the SVD algorithm to run. Fortunately, Incremental PCA (IPCA) algorithms have been developed: you can split the training set into mini-batches and feed an IPCA algorithm one minibatch at a time. This is useful for large training sets, and also to apply PCA online (i.e., on the fly, as new instances arrive).
7.29. Randomized PCA¶
Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic algorithm that quickly finds an approximation of the first d principal components. Its computational complexity is \(O(m \times d^2)+O(d^3)\), instead of \(O(m \times n^2) + O(n^3)\), so it is dramatically faster than the previous algorithms when \(d\) is much smaller than \(n\).
7.30. Kernel PCA¶
The kernel trick is a mathematical technique that implicitly maps instances into a very high-dimensional space (called the feature space), enabling nonlinear classification and regression with Support Vector Machines. Recall that a linear decision boundary in the high-dimensional feature space corresponds to a complex nonlinear decision boundary in the original space. It turns out that the same trick can be applied to PCA, making it possible to perform complex nonlinear projections for dimensionality reduction. This is called Kernel PCA (kPCA). It is often good at preserving clusters of instances after projection, or sometimes even unrolling datasets that lie close to a twisted manifold. For example, the following code uses Scikit-Learn’s KernelPCA class to perform kPCA with an
from sklearn.decomposition import KernelPCA
rbf_pca = KernelPCA(n_components = 2, kernel="rbf", gamma=0.04)
X_reduced = rbf_pca.fit_transform(X)
7.31. LLE¶
Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction (NLDR) technique. It is a Manifold Learning technique that does not rely on projections like the previous algorithms. In a nutshell, LLE works by first measuring how each training instance linearly relates to its closest neighbors (c.n.), and then looking for a low-dimensional representation of the training set where these local relationships are best preserved (more details shortly).
7.32. Other techniques¶
There are many other dimensionality reduction techniques, several of which are available in Scikit-Learn.
Here are some of the most popular:
Multidimensional Scaling (MDS) reduces dimensionality while trying to preserve the distances between the instances.
Isomap creates a graph by connecting each instance to its nearest neighbors, then reduces dimensionality while trying to preserve the geodesic distances between the instances.
t-Distributed Stochastic Neighbor Embedding (t-SNE) reduces dimensionality while trying to keep similar instances close and dissimilar instances apart. It is mostly used for visualization, in particular to visualize clusters of instances in high-dimensional space (e.g., to visualize the MNIST images in 2D).
Linear Discriminant Analysis (LDA) is actually a classification algorithm, but during training it learns the most discriminative axes between the classes, and these axes can then be used to define a hyperplane onto which to project the data. The benefit is that the projection will keep classes as far apart as possible, so LDA is a good technique to reduce dimensionality before running another classification algorithm such as a Support Vector Machine (SVM) classifier discussed in the SVM lectures.


| Interface | Description |
|---|---|
| ConfigLoader | |
| DevMetaProvider | |
| EntityPrincipal | |
| EntitySubject | |
| KernelHandler | |
| LogHandler | |
| NetworkedApplication |
Represents Applications that implements some form of TCP/IP networking.
|
| RegistrarBase |
Used to track event and task registration.
|
| VendorRegistrar |
| Class | Description |
|---|---|
| BaseApplication |
When a
BaseApplication is instigated, its main thread is dedicated to
running the Looper that takes care of managing the top-level application tasks and parcels. |
| ClassBasedRegistrar | |
| ConfigData | |
| ConfigRegistry | |
| ConfigRegistry.ConfigKeys | |
| DefaultApplication |
Implements a basic application environment with modules Config, Tasks, Events.
|
| Env | |
| Foundation |
Used for accessing majority of the Foundation API.
The first call to this class MUST be the thread that will initiate the main loop. |
| Foundation.ConfigKeys | |
| FoundationLooper | |
| Kernel | |
| Kernel.Logger | |
| MetaMap | |
| MinimalApplication | |
| PropDevMeta | |
| VendorMeta |
| Enum | Description |
|---|---|
| ConfigLoader.CommitType | |
| Runlevel |
| Exception | Description |
|---|---|
| FoundationCrashException |
INTERNAL USE ONLY
You should attempt to caught and mute this exception within your main() class.
|
| Annotation Type | Description |
|---|---|
| Hook | |
| ParameterClass |
Specifies the class used for a method/constructor parameter.
|
| ProvidesClass |
Annotates methods and fields from within binding provider class to indicate the full provided namespace.
|
| Singular |
Used to label a class as only having one singular instance allowed, all other classes can be instigated as many times as requested.
|


"MemoryBanks.exe"
Readme
Introduction
The MemoryBanks.exe is an example program that allows the user to
read/write
iButton/1-Wire devices supported by the OW.NET API's "Memory Bank"
object. It will write only to a "General Purpose"
Memory Bank if present.
This particular example was written in Microsoft Visual Basic .NET with Microsoft Visual Studio .NET 2005.
Requirements
1-Wire Drivers
version 4.00
or higher, with the OneWireAPI.NET.dll in the same directory as the
executable.
.NET Framework 2.0 Redistributable:
Visual J# .NET 2.0 Second Edition Redistributable:
When compiling this program, make sure that the program references
“vjslib”, and the OneWireAPI.NET.dll.
Before running this program, make sure to set the default 1-Wire adapter settings to the preferred type (usually "{DS9490}") and number (usually "USB1"). This can be done by running the "Default 1-Wire NET.exe" program found in the 1-Wire Drivers application directory.
Operation
To run
the
MemoryBanks.exe program, double-click on the "MemoryBanks.exe"
file. This runs the program. The main window of the program
will
appear containing a text box and 2 buttons. The top of the text
box shows
which 1-Wire adapter and port is being used as the "Default" adapter
and port. To change the "Default" adapter and port, please look
in the 1-Wire Drivers folder under "Program Files" and find the
"Default 1-Wire Net" icon. Double-clicking this icon will allow
the user to select from the different available 1-Wire adapters and
ports.
The two buttons on the main window of MemoryBanks.exe are titled "Read Memory Banks" and "Write 'Testing 123' to Each Main Memory Bank Page". Clicking on "Read Memory Banks" will read out the contents of each Memory Bank on every iButton/1-Wire device found on a 1-Wire network. Please note that some 1-Wire devices do not contain any Memory Banks while others may contain multiple Memory Banks.
The second button is the "Write 'Testing 123' to Each General Purpose Memory Bank Page" button. This will write to every page of a "General Purpose" Memory Bank of a 1-Wire device. Optionally, the programmer can make a single line source code change to write a page packet instead (consists of a length byte, followed by "Testing 123" in byte format, followed by a 2-byte CRC16). Also, a way to write raw bytes through .NET's System.Buffer namespace is documented (but commented out). It shows how to write unsigned byte information (for example, 0xFF) to an SByte (a widely used data type for sending data to and from 1-Wire devices through .NET).
Note: Please be patient when using the DS1977, as the program will read/write the entire 32K of memory contents.
Files
AssemblyInfo.vb
- VB.NET assembly file.
FormMemBanks.resx - VB.NET resource file.
FormMemBanks.vb - MemoryBanks.exe main form.
MemoryBanks.sln - VB.NET MemoryBanks solution file (opens project solution)
MemoryBanks.vbproj - VB.NET project file.
MemoryBanks.vbproj.user - VB.NET project user file.
Readme.html - This file.
..\..\..\..\Lib\OW.NET\OneWireAPI.NET.dll - 1-Wire API
..\bin\MemoryBanks.exe - Program executable


Health Monitor
High Level Summary
Detailed Testing Logs
BCH-API Integration tests
rest.bicoin.com Integration tests
Real-Time System Monitoring
Summary
sdb disk
CPUs


Here you can find out how to get to your ID card. What kind of documents do you need to get an ID card? Which competent authorities need to contact, how long it takes to wait for it to get our ID card ....
Click to view infoHere you can find information about what documents you need to get a passport.Here are answers to all questions concerning passports, such as: how to get a passport, to whom to address if I lose it ...
Click to view infoYou want to travel to another state, but you do not know how to get a visa, we are here to give you all the answers regarding obtaining visas, as well as other issues related to embassies in our country.
Click to view infoDocuments that you need to get a driver's license are, in addition to the certificate of passed driving exam and payment slips, personal documents and medical certificates.
Click to view infoHere you can expect news and events from our city.
Far far away, behind the word mountains, far from the countries Vokalia and Consonantia, there live the blind texts.
Far far away, behind the word mountains, far from the countries Vokalia and Consonantia, there live the blind texts.
These are the girls behind this site. in partnership with Girls Coding Kosova.
Far far away, behind the word mountains, far from the countries Vokalia and Consonantia, there live the blind texts.
Far far away, behind the word mountains, far from the countries Vokalia and Consonantia, there live the blind texts.
Far far away, behind the word mountains, far from the countries Vokalia and Consonantia, there live the blind texts.
Far far away, behind the word mountains, far from the countries Vokalia and Consonantia, there live the blind texts.


Toggle navigation
Can you vote
Search
Part
Section
Serial
EPIC/Voter ID
Status
Not Able to Find the VOTER ID, Please cross check the ID entered above


Mobile: +91<PHONE_NUMBER>(Vishnu varun Babu) Mobile: +91<PHONE_NUMBER>(Deepika Tamizhvanan)
<EMAIL_ADDRESS>www.indcon17.com
facebook
College Of Engineering
Guindy


Ernie Els set for 2020 PGA Tour Champions debut
Cape Town - Four-time major champion Ernie Els will make his debut on the 2020 PGA Tour Champions.
The PGA Tour Champions is a membership organisation of professional golfers aged 50 and older, and includes 33 members of the World Golf Hall of Fame.
The PGA Tour Champions announced on Wednesday that the 2020 schedule will feature 27 events in five countries with a total prize money of nearly $59 million.
Els, who is the captain of the International Presidents Cup team, celebrated his 50th birthday on October 17.
The 'Big Easy' will join fellow Hall of Famer Phil Mickelson, 17-time PGA Tour winner Jim Furyk and 2003 Masters champion Mike Weir, who will all turn 50 in the next 12 months.
The 2020 season will begin with Els' debut at the Mitsubishi Electric Championship at Hualalai the week of January 16-18 (Saturday finish), where American Tom Lehman will be the defending champion.
"I am looking forward to my first start on PGA Tour Champions in January at the Mitsubishi Electric Championship in Hawaii," said Els in a press release.
"It will be great to reconnect with all of my friends I grew up competing with, and I know the camaraderie and competition will provide a lot of excitement for our fans."
The schedule of five major championships starts in May with the Regions Tradition in Birmingham, Alabama, while the season's other Majors will be at the Harbor Shores Golf Club in Michigan (Senior PGA Championship), Newport Country Club (US Senior Open), Firestone Country Club (Senior Players Championship) and Sunningdale Golf Club in England (Senior Open).
On Thursday, captain Els made his four International wildcard picks for December's Presidents Cup, which will be held at Royal Melbourne Golf Club in Australia.
Els will skipper the International team's challenge against the US team led by captain Tiger Woods.
- Compiled by Lynn Butler


István Merza is your System Admininstrator.
whoami
* István Merza, Sörfutár * Information Systems Engineer * IT Administrator, sysadmin * Identity&Access Management * Based in Munich, Germany * Hungarian Monty Python fan * Liquor of Choice: BEER
why this site
* online presence * experiment * learn * for professional inquiries please use LinkedIn
funstuff
* SysAdminDay!


We are looking for people who care for their loved ones (mom, dad, aunt, spouse etc.) to take part in a research study. The focus of this study will be on your caring habits. We aim to understand how technology can be used by person who cares for their loved ones to better respond and manage the needs of elderly people in a more efficient and effective way. With your permission, we will be looking forward to gain a better understanding of your habits, needs, and the barriers you encounter during your work as well as the strategies you use to overcome them.
If you participate you will be invited for an interview (Phase 1) to get a sense of your caring process (the interview could be over Skype or face-to-face depending on your preferences). After the interview, you will be asked to participate in a diary study (Phase 2) for a week which should take a maximum of 10 minutes each day. At the end of the study you will be invited to take part in a Focus group (Phase 3) in London where you will be provided a chance to communicate with other participants in this study to exchange views together.
All the phases mentioned above would be organized at your convenience. It will be up to you to decide to take part in either of the phases. During this study, you will not be asked to provide any access to the people you care for.
This study is being conducted by researchers from University College London, UK. The study has been approved by the UCLIC Research Department’s Ethics Chair [Project ID No]: UCLIC/1415/011/MSc Berthouze/Das/Anand.
Please note that by completing this form you are giving us permission to contact you in relation to this study. If you are chosen to participate in the study we will contact you via email to arrange a date and time for the interview at your convenience. Information gained in interviews will be used for the purpose of academic research and may be presented in an academic publication and/or conferences, workshops and/or teaching material. Confidentiality and anonymity will be maintained, and it will not be possible to identify you from any such publications. During the interview, an audio recording will be made for later transcription. Your details will NOT be used for any purposes other than this. You are free to withdraw from the study at any time.
Note: All data will be collected and stored in accordance with the Data Protection Act 1998 and will be kept anonymous.
If you have any questions or encounter any problems you may contact us at<EMAIL_ADDRESS>Principal Researcher of the study: Dr. Nadia Bianchi-Berthouze , Reader in Affecting Computing, UCL Interaction Centre
The growth of an ageing population and people with long-term conditions is a challenge to the current health care strategy. King fund organization’s [Appleby 2013] published report points out, the people with long-term conditions, i.e., around 15 million, will rise by a third in the next 10 years. Users with long-term conditions will account for more than 70% of inpatient beds and half of all general practitioners (GP) appointments in England [Wilson et al., 2013].


index.html --- layout: home author_profile: true sidebar: name : "Daniel Downs" avatar : "assets/downs photo cropped.png" bio : "Hello. Welcome to the personal website of Dan Downs." location : "New York, NY" email<EMAIL_ADDRESS>links: - label: "Email" icon: "fas fa-fw fa-envelope-square" email<EMAIL_ADDRESS>- label: "Twitter" icon: "fab fa-fw fa-twitter-square" url: "https://twitter.com/DanDownsStartup" - label: "GitHub" icon: "fab fa-fw fa-github" url: "https://github.com/danielrdowns/" - label: "Instagram" icon: "fab fa-fw fa-instagram" url: "https://instagram.com/danielrdowns" - label: "LinkedIn" icon: "fab fa-fw fa-linkedin" url: "https://www.linkedin.com/in/danielrdowns/" nav: "docs" ---
Hello.
Welcome to the personal website of Dan Downs.
Please click around to explore.


Start here
Get started with Media right now
Use the tool below to upload your images or documents to a media folder.
Follow these steps:
- Click Install and follow the on screen instructions to install the Desktop Media Uploader
- Enter your login details for the site and click Sign In
- Choose a media folder to upload files to from the Upload files to... dropdown list
- Drag the files and folders you wish to upload directly into the Desktop Media Uploader application
- Click Upload to start uploading
For a more thorough guide on how to use the Desktop Media Uploader, checkout this video.


OFF ROAD is a guild that can improve your skill and eard many benefit from team works. guild that profide any role that you master and have many friends there. if you are interested in joining our guild, so lets join and get many benefit form team works
this site is made for you who wants to improve your skill and earn your reward from job that posted from others. and you also can post a job or do a party to compleate the job that requested.
Zee
Andy
Elang
Nick, Tink, adn Pap Mick