Scientific Data Has Become So Complex, We Have to Invent New Math to Deal With It - Wired Science
Simon DeDeo, a research fellow in applied mathematics and complex systems at the Santa Fe Institute, had a problem. He was collaborating on a new project analyzing 300 years’ worth of data from the archives of London’s Old Bailey, the central criminal court of England and Wales. Granted, there was clean data in the usual straightforward Excel spreadsheet format, including such variables as indictment, verdict, and sentence for each case. But there were also full court transcripts, containing some 10 million words recorded during just under 200,000 trials.
How the hell do you analyze that data?” DeDeo wondered. It wasn’t the size of the data set that was daunting; by big data standards, the size was quite manageable. It was the sheer complexity and lack of formal structure that posed a problem. This “big data” looked nothing like the kinds of traditional data sets the former physicist would have encountered earlier in his career, when the research paradigm involved forming a hypothesis, deciding precisely what one wished to measure, then building an apparatus to make that measurement as accurately as possible.

Scientific Data Has Become So Complex, We Have to Invent New Math to Deal With It - Wired Science

Simon DeDeo, a research fellow in applied mathematics and complex systems at the Santa Fe Institute, had a problem. He was collaborating on a new project analyzing 300 years’ worth of data from the archives of London’s Old Bailey, the central criminal court of England and Wales. Granted, there was clean data in the usual straightforward Excel spreadsheet format, including such variables as indictment, verdict, and sentence for each case. But there were also full court transcripts, containing some 10 million words recorded during just under 200,000 trials.

How the hell do you analyze that data?” DeDeo wondered. It wasn’t the size of the data set that was daunting; by big data standards, the size was quite manageable. It was the sheer complexity and lack of formal structure that posed a problem. This “big data” looked nothing like the kinds of traditional data sets the former physicist would have encountered earlier in his career, when the research paradigm involved forming a hypothesis, deciding precisely what one wished to measure, then building an apparatus to make that measurement as accurately as possible.

Notes

  1. axtigo reblogged this from smarterplanet
  2. wordythings reblogged this from postmortemdecay666
  3. littlebluhouse reblogged this from we-are-star-stuff
  4. amateur-american-eolai reblogged this from we-are-star-stuff
  5. computationalsociology reblogged this from smarterplanet
  6. gotsharpies reblogged this from dstrichit
  7. viirulentscience reblogged this from we-are-star-stuff
  8. theperfectionistjournalist reblogged this from smarterplanet
  9. awesome-brick reblogged this from smarterplanet
  10. squirrelonsquirrel reblogged this from chrodgangsta
  11. x-chimera reblogged this from smarterplanet
  12. leftismisthebestism reblogged this from smarterplanet
  13. girlslovescience reblogged this from scienceing and added:
    Big data gonna change the world.
  14. jasonvsasha reblogged this from smarterplanet and added:
    New math in the making…
  15. peterpositron reblogged this from we-are-star-stuff
  16. electrumplated reblogged this from scienceing
  17. lightninging reblogged this from scienceing
  18. gingerinagreysuit reblogged this from scienceing
  19. qswoas reblogged this from smarterplanet
  20. frostedquill reblogged this from scienceing
  21. avocadoshirts reblogged this from scienceing
  22. apopyllus-now reblogged this from scienceing
  23. eternalstruggleofrhubarbpie reblogged this from scienceing
  24. rebelgoatalliance reblogged this from scienceing
  25. scienceing reblogged this from smarterplanet
  26. spencella reblogged this from we-are-star-stuff

Recent comments

Blog comments powered by Disqus