Scientific Data Has Become So Complex, We Have to Invent New Math to Deal With It - Wired Science
Simon DeDeo, a research fellow in applied mathematics and complex systems at the Santa Fe Institute, had a problem. He was collaborating on a new project analyzing 300 years’ worth of data from the archives of London’s Old Bailey, the central criminal court of England and Wales. Granted, there was clean data in the usual straightforward Excel spreadsheet format, including such variables as indictment, verdict, and sentence for each case. But there were also full court transcripts, containing some 10 million words recorded during just under 200,000 trials.
How the hell do you analyze that data?” DeDeo wondered. It wasn’t the size of the data set that was daunting; by big data standards, the size was quite manageable. It was the sheer complexity and lack of formal structure that posed a problem. This “big data” looked nothing like the kinds of traditional data sets the former physicist would have encountered earlier in his career, when the research paradigm involved forming a hypothesis, deciding precisely what one wished to measure, then building an apparatus to make that measurement as accurately as possible.

Scientific Data Has Become So Complex, We Have to Invent New Math to Deal With It - Wired Science

Simon DeDeo, a research fellow in applied mathematics and complex systems at the Santa Fe Institute, had a problem. He was collaborating on a new project analyzing 300 years’ worth of data from the archives of London’s Old Bailey, the central criminal court of England and Wales. Granted, there was clean data in the usual straightforward Excel spreadsheet format, including such variables as indictment, verdict, and sentence for each case. But there were also full court transcripts, containing some 10 million words recorded during just under 200,000 trials.

How the hell do you analyze that data?” DeDeo wondered. It wasn’t the size of the data set that was daunting; by big data standards, the size was quite manageable. It was the sheer complexity and lack of formal structure that posed a problem. This “big data” looked nothing like the kinds of traditional data sets the former physicist would have encountered earlier in his career, when the research paradigm involved forming a hypothesis, deciding precisely what one wished to measure, then building an apparatus to make that measurement as accurately as possible.

Notes

  1. sadparabola reblogged this from autisticbuckybarnes
  2. axtigo reblogged this from smarterplanet
  3. wordythings reblogged this from postmortemdecay666
  4. littlebluhouse reblogged this from we-are-star-stuff
  5. amateur-american-eolai reblogged this from we-are-star-stuff
  6. computationalsociology reblogged this from smarterplanet
  7. gotsharpies reblogged this from dstrichit
  8. viirulentscience reblogged this from we-are-star-stuff
  9. theperfectionistjournalist reblogged this from smarterplanet
  10. awesome-brick reblogged this from smarterplanet
  11. nosquirrelno reblogged this from chrodgangsta
  12. verklepmt reblogged this from smarterplanet
  13. communerd reblogged this from smarterplanet
  14. girlslovescience reblogged this from scienceing and added:
    Big data gonna change the world.
  15. jasonvsasha reblogged this from smarterplanet and added:
    New math in the making…
  16. peterpositron reblogged this from we-are-star-stuff
  17. electrumplated reblogged this from scienceing
  18. lightninging reblogged this from scienceing
  19. gingerinagreysuit reblogged this from scienceing
  20. qswoas reblogged this from smarterplanet
  21. frostedquill reblogged this from scienceing
  22. avocadoshirts reblogged this from scienceing
  23. apopyllus-now reblogged this from scienceing
  24. eternalstruggleofrhubarbpie reblogged this from scienceing
  25. rebelgoatalliance reblogged this from scienceing
  26. scienceing reblogged this from smarterplanet

Recent comments

Blog comments powered by Disqus