[Shared] HOW BIG IS TOO BIG FOR JSON?

Table of Contents

When To Use Pagination in REST Resources” and advocated for better UX Design instead of pagination. This generated another question: “Just how big is TOO BIG for JSON? In the context of a desktop browser, there are many factors to answer this question that include file, memory and object size as well as transport and loading time. I wanted find the “sweet spot” for speaking to the maximum usable size of a JSON object. So I decided to test desktop browsers to see how they handle JSON at different sizes. At some point, I would also like to see how this translates to mobile devices. The JSON samples were pulled from customer data in sizes ranging from 1 record to 1,000,000 records. Each record averages around 200 bytes a piece. The core HTML file only loads the JSON file and sets it to the testData variable. The following is an example JSON object used in this test:

Five (5) samples per browser were taken to create the averages. Originally ten (10) samples per browser were taken, but it quickly became obvious that the data wasn’t changing enough to make the extra effort necessary. The results are using the operating system’s process monitors and each browser’s developer tools. Since network traffic can be deceiving, I chose to load the JSON file from the local file system and provide estimated download times from Numion’s Download Time Calculator at http://www.numion.com/calculators/time.html. Memory usage was taken after the object loaded, while the browser was at rest. Each sample was taken from a fresh process. Memory was taken from each browsers “tab” or renderer process, with the exception of Firefox which was taken from the single core process. These tests were run on my 2011 2.2GHz i7 MacBook Pro with 8GB of RAM. During the test, I spent some time seeing how usable each browser was while loading the object. I wanted to see if it was possible to load large objects while still keeping the browser usable and how the browser performed after the object was loaded. Most of the “During Load” results are untestable as I am not quick enough to tell if the browser is responsive in 200ms. The following are the results from the tests:
Records Download Time* Chrome (OS X) Firefox (OS X) Safari (OS X) IE9 (Win 7)
1,000,000 (153.37MB) 19:51 DOM Ready:16.4s RAM: 1.09GB Loading:browser hang Loaded:usable, 1 tab crash DOM Ready:14.29s RAM: 1.82GB Loading:untestable Loaded:usable DOM Ready:5.13s RAM: 1.76GB Loading:browser hang Loaded: usable Data failed to load and browser became unresponsive.
750,000 (115.13MB) 14:54 DOM Ready:12.24s RAM: 727.3MB Loading:browser hang Loaded:usable DOM Ready:13.8s RAM: 1.48GB Loading:browser hang Loaded:usable DOM Ready:3.82s RAM: 1.39GB Loading: barely usable Loaded: usable Data failed to load and browser became unresponsive.
500,000 (76.69MB) 9:56 DOM Ready:9.13s RAM: 512.0MB Loading:browser hang Loaded:usable DOM Ready:12.19s RAM: 1.14G Loading:browser hang Loaded:usable DOM Ready:2.49s RAM: 1.02GB Loading: usable Loaded: usable Data failed to load and browser became unresponsive.
250,000 (38.44MB) 4:58 DOM Ready:2.98s RAM: 289.8MB Loading:barely usable Loaded:usable DOM Ready:3.09s RAM: 659.3MB Loading:barely usable Loaded:usable DOM Ready:1.57s RAM: 550.0MB Loading: usable Loaded: usable DOM Ready:4.88s RAM: 537.1MB Loading:browser hang Loaded: usable
100,000 (15.5MB) 2:00 DOM Ready:1.24s RAM: 150.4MB Loading:usable Loaded:usable DOM Ready:1.62s RAM: 424.0MB Loading:usable Loaded:usable DOM Ready:463ms RAM: 296.2MB Loading:untestable Loaded: usable DOM Ready:1.71s RAM: 210.6MB Loading:untestable Loaded: usable
50,000 (7.77MB) 1:00 DOM Ready:521ms RAM: 89.7MB Loading:untestable Loaded:usable DOM Ready:1.08s RAM: 308.9MB Loading:untestable Loaded:usable DOM Ready:243ms RAM: 192.8MB Loading:untestable Loaded: usable DOM Ready:801ms RAM: 110.1MB Loading:untestable Loaded: usable
25,000 (3.87MB) 00:30 DOM Ready:255ms RAM: 67.5MB Loading:untestable Loaded:usable DOM Ready:646ms RAM: 269.3MB Loading:untestable Loaded:usable DOM Ready:169ms RAM: 130.8MB Loading:untestable Loaded: usable DOM Ready:408ms RAM: 51.1MB Loading:untestable Loaded: usable
10,000 (1.55MB) 0:12 DOM Ready:144ms RAM: 53.1MB Loading:untestable Loaded:usable DOM Ready:193ms RAM: 234.7MB Loading:untestable Loaded:usable DOM Ready:66ms RAM: 105.1MB Loading:untestable Loaded: usable DOM Ready:117ms RAM: 35.1MB Loading:untestable Loaded: usable
5,000 (796.57KB) 0:06 DOM Ready:81ms RAM: 39.14MB Loading:untestable Loaded:usable DOM Ready:123ms RAM: 220.9MB Loading:untestable Loaded:usable DOM Ready:42ms RAM: 77.2 Loading:untestable Loaded: usable DOM Ready:151ms RAM: 18.2MB Loading:untestable Loaded: usable
1,000 (159.59KB) 0:01 68ms RAM: 29.1MB Loading:untestable Loaded:usable 76ms RAM: 209.2MB Loading:untestable Loaded:usable 41ms RAM: 72.0 Loading:untestable Loaded: usable 63ms RAM: 11.3MB Loading:untestable Loaded: usable
500 (80.01KB) 0:00 DOM Ready:61ms RAM: 32.5MB Loading:untestable Loaded:usable DOM Ready:59ms RAM: 209.2MB Loading:untestable Loaded:usable DOM Ready:32ms RAM: 71.6MB Loading:untestable Loaded: usable DOM Ready:54ms RAM: 8.6MB Loading:untestable Loaded: usable
250 (40.05KB) 0:00 DOM Ready:55ms RAM: 28.7MB Loading:untestable Loaded:usable DOM Ready:53ms RAM: 209.1MB Loading:untestable Loaded:usable DOM Ready:33ms RAM: 71.4MB Loading:untestable Loaded: usable DOM Ready:62ms RAM: 7.1MB Loading:untestable Loaded: usable
100 (15.97KB) 0:00 DOM Ready:53ms RAM: 28.8MB Loading:untestable Loaded:usable DOM Ready:66ms RAM: 209.1MB Loading:untestable Loaded:usable DOM Ready:33ms RAM: 70.1MB Loading:untestable Loaded: usable DOM Ready:39ms RAM: 6.6MB Loading:untestable Loaded: usable
50 (8.05KB) 0:00 DOM Ready:40ms RAM: 27.9MB Loading:untestable Loaded:usable DOM Ready:68ms RAM: 209.1MB Loading:untestable Loaded:usable DOM Ready:36ms RAM: 71.1MB Loading:untestable Loaded: usable DOM Ready:35ms RAM: 5.7MB Loading:untestable Loaded: usable
25 (4.07KB) 0:00 DOM Ready:12ms RAM: 28.6MB Loading:untestable Loaded:usable DOM Ready:77ms RAM: 209.1MB Loading:untestable Loaded:usable DOM Ready:29ms RAM: 71MB During Load: untestable Loaded: usable DOM Ready:29ms RAM: 3.5MB Loading:untestable Loaded: usable
1 (192B) 0:00 DOM Ready:7ms RAM: 28.2MB Loading:untestable Loaded:usable DOM Ready:59ms RAM: 207MB Loading:untestable Loaded:usable DOM Ready:35ms RAM: 64.0MB Loading:untestable Loaded: usable DOM Ready:19ms RAM: 3.1MB Loading:untestable Loaded: usable
* Download time is estimated using Numion’s Download Time Calculator, based on 1.544Mbps connection in (mm:ss) format. As you can see from my results, the browsers were all surprisingly close to one another. The most interesting part for me was that Internet Explorer failed to load the object after the 250,000 record test. I couldn’t find anything pointing to a size limitation here. If anyone knows as to why IE failed to load the 500,000 record object, let me know. From this test, I am considering the sweet spot to be around 10,000 records at (1.55MB). The maximum number of usable records I would push to a browser would be around 25,000 records (3.87MB). Keep in mind there are numerous factors to keep in mind when determining how many records you should return to your JavaScript application. The purpose of this test was to help identify a general maximum number for conversations around large record sets with JSON. -- Nguồn: https://joshzeigler.com/technology/web-development/how-big-is-too-big-for-json]]>

Leave a Reply

Your email address will not be published. Required fields are marked *