Kiến Thức Chung

Let’s clear up the confusion regarding storage in Android once and for all, including adoptable storage in Marshmallow. : Android

Summary (read the gory details for the whys and hows) Basically, users have access to a single folder, /storage/emulated. It’s the one that contains directories you should be familiar with like ‘DCIM’, ‘Music’, ‘Downloads’. This folder may point to your true internal storage or an SD card in adoptable storage mode. Unfortunately, this folder is called different things like “sd card” due to history of Android which is what causes confusion; just ignore what apps choose to call this “home directory” for now. SD cards or OTG drives before Lollipop, called “portable mode” in Marshmallow function more like a removable flash drive with your laptop: they’re not considered your “home” folder and apps often don’t even offer the option to read/write to portable storage. In general, it should be fine to leave the SD card in portable mode if you have 32 GB+ of internal storage.

If you have less than 32 GB, you may consider adoptable storage. The caveat though is you’ll either see your internal storage or the adopted SD card but never both simultaneously. This may seem like worthless but remember apps can continue to be individually placed on internal storage or on the adopted SD card. This arrangement can be simple to think about if you consider your internal storage to be your home for your apps and your giant adopted SD card to contain your files (along with any exceptionally large apps). Contrary to what everyone likes to say, Google implemented Adoptable storage pretty well because you can’t simply merge two storage devices without risking the integrity of the filesystem. It’s not slow, either, at least on a mid-range device like Moto G. It doesn’t feel slow on normal apps. Large games/app placed on adopted storage may load a little slower. User choice is good; adoptable storage shouldn’t be removed from any device, imo. Make the option well hidden, maybe, but don’t remove it.

 

Gory details First of all, I want to say at the outset that the Android storage system is a tragedy. It’s a miracle it somehow still works today. There are too many partitions, too many folders, and too many locations to store files. Android doesn’t even break down your used storage properly – my used space + my free space never adds up to the total available space. There have been more breaking API changes in the Android SDK regarding storage than I can keep track of.

Naturally, there is a lot of confusion regarding storage in r/Android. With adoptable storage to add to the confusion, some of the things people say are downright wrong. I wanted to make a thread that presents the facts once and for all. I’ll explain in terms of how it works on my Moto G 2015 (XT1550). It has a real SD card that I’ve been using for adoptable storage for the past two months. I’ve rooted my device for exploration purposes, but I’ll limit my explanation to how it would behave on a stock device. I’m reasonably confident in the overall picture I’ve gathered, but I realize my knowledge incomplete, so I welcome any corrections.

My phone has 16 GB of storage. I suspect this actually means 16 billion bytes 15,758,000,128 bytes of storage (I checked), which translates to only 14.7 GB since the OS uses 1024 for the conversions (1024 bytes = 1 KB, 1024 KB = 1 MB and so on…). Henceforth, I will refer to storage when calculated based on 1024 as MiB and GiB, etc…

Of the 14.7 GiB, only 11.92 GB is available to the user, though. This is because the overall storage of the device is sectioned off into many partitions (there are 42!!!), only one of which you, the user, directly has access to (the /data one). I’ll list some of the important partitions:

  • /system – 2.3 GiB (ext4 filesystem): This is the “ROM” of the device. It basically contains the Android OS and is the bulk of what’s updated when you get software updates. It is mounted as read-only and can only be officially written to by an update that is cryptographically signed by Motorola (unless you “unlock” your bootloader).

  • /cache – 256 MiB (ext4 filesystem): This, as the name implies, is a partition used to store temporary files. Note that this is different from cache files stored by apps, however. When you look in your storage break down in the settings screen of Android, the cache used by applications is not this partition. That lives in the /data partition. /cache is not even readable by applications. I suspect it’s used by the system for its own purposes. Would appreciate any explanations you guys can offer for this one.

  • /data – 11.9 GiB (Flash Friendly filesystem): The is the userdata partition. It’s what the user can mainly manipulate. It contains your user-installed apps (and the updates to the system apps) and almost all your app data. I’ll get to that almost part in a bit. Data that an app stores inside your /data is guaranteed by the OS to remain private, in the sense that no other app or the user can access it on an unmodified device (disregarding the ADB backup/restore feature for apps and data). See appendix I regarding all the stuff inside /data. The part important for this guide is that there is just one folder – /data/media/0 – that is exposed to the user and apps as the “usb storage” or “internal storage” or “internal sd card”. It’s the root of the folders we’ve come to know and love like DCIM, Music, Downloads, etc… Files stored by the system under /data that are not under /data/media are not visible to users. You can only reclaim that space by a factory reset. I’m still looking at what kinds of logs might be stored under /data and how quickly they grow, consuming precious internal storage.

Xem thêm:   Thiết kế M&E nhà phố – Bài 3: Tạo file AutoCAD mẫu chuẩn vẽ điện nhà phố

Xem thêm :  Hướng dẫn cách chơi vị tướng rouie cực mạnh

Whew! They add up to roughly 14.5 GiB, just 200 MiB shy of my device’s 14.9 GiB. It doesn’t stop here, sadly. There are more partitions, such as the cell modem, fsg and more that each take a (small) slice out of that remaining 200 MiB. I’m not sure what they all do but there are a total of 42 of them! An interesting partition is /persist. While I’m not certain about this, I suppose this partition is what allows features like device protection to work even across factory resets. Factory reset only wipes the /data partition, so /persist, well, persists. Users and apps, as expected, can’t access /persist on unrooted devices though. I hope this part shows users why they have less space than the advertised amount. It is not “lost due to formatting” – that is only tens of MiBs at most. It is mostly a little shady advertising carried over from the HDD days of computing and the fact that some of your device’s space is reserved for other partitions – you never even see those files.

SD Cards (Portable Mode in Marshmallow) As you may have noticed, I haven’t said a thing about SD cards yet. This is because they are treated completely differently. When I use my Moto G without a physical SD card in the device, the OS still simulates an “internal” SD card. This “internal” SD card is accessed via the /storage/emulated directory (which contains numbered directories for eac user). So a file manager is actually browsing /storage/emulated/0 when it’s looking at user storage. This directory actually is backed by data in /data/media – /storage/emulated is just a mount point. When I do put in a physical SD card, it is mounted at /storage/53D-DS32 where 53D-DS32 is just some unique identifier for the card. User-installed apps can not write any storage outside /storage/emulated/0 so when you want to capture pictures to your external SD card using a downloaded camera app, you have to “select the SD card” in an additional prompt – this is since KitKat Lollipop (thanks) I believe. When you use an SD card in portable mode, you essentially have two locations to store user data: /storage/emulated/0 (the internal storage) and /storage/53D-DS32 (your SD card). This is a CHORE to maintain. I hated it. You’re always wondering whether this application is storing data on my internal storage or on my SD card. And applications usually don’t even offer you the option to decide where to store user data (I’m looking at you, WhatsApp). The only reasonable way to use an SD card in portable mode on Android is just use it for Pictures and Music. If your device has limited internal storage and you want to download lots of apps, tough. Your SD card wouldn’t help you before Marshmallow (see Appendix 3 for a severely crippled version of “move app to SD” Android had before Marshmallow). Apps can no longer be moved to the SD card if you’re using it in Portable Mode in Marshmallow.

edit: As /u/alexfrias points out, some OEMs are allowing apps to be moved to SD in portable mode on Marshmallow. So, it depends on your device. It seems like since both Moto phones and Android One phones don’t support moving apps to SD with portable storage, this is how it is in vanilla Android. Also, files on portable storage are visible to all users on the device. Files on internal storage or adopted internal storage are private to each user.

Adoptable storage When you adopt an SD card as internal storage, however, all hell breaks loose. No, I’m just kidding it’s actually quite simple. The user is promising Android that “I won’t remove this SD card from the device; please integrate it with your internal storage. I understand that if I remove the card, I don’t expect my usb storage data to be there or some apps to work properly”.

But, the physical reality is that the SD card CAN be removed and SD cards do fail prematurely sometimes. So, Google couldn’t actually just combine the SD card with your device’s internal storage. It wouldn’t work. Think about it. If the card failed or a curious user removed the card just to see what happened (that would be me), NO file under /data (your userdata paritition, remember) would be guaranteed to be available. In fact, removing the SD card could cause such file corruption, that the only solution would be wiping the filesystem under /data and starting over (factory resetting the device). Clearly, this is not a viable route. Your PHONE couldn’t even be guaranteed to be stable enough to call 911 this way if the SD card failed just at the wrong time.

The only other way to “adopt” the SD card was to keep its filesystem separate from /data and offer the user choice of where to store what. This is exactly what Google has done. They give you an option to keep apps (everything in /data/app/package.name) and apps’ private data (/data/data/package.name) on internal storage or the adopted SD card. One thing that I don’t like is that ALL apps are not movable to the adopted SD card still. No system app can be moved, which makes sense. But even for user apps, it seems like the developer has to mark the app as “movable to SD” in the same as they used to for the Move to SD feature until Lollipop. It’s opt-in, basically, from the developer’s side. I don’t like this, because the user is already promising not to remove the SD card. They’ll understand if apps are unavailable/don’t work properly without their adopted SD card.

With the “usb storage” “user-accessible storage” (the /storage/emulated folder with DCIM, Music, etc folders) aspect of your data, Google did something reasonable. Instead of forcing the user to pick for EVERY file whether they want it stored in the device’s internal storage or the adopted storage, they simply made the internal memory unavailable for storage. That is, /storage/emulated is mounted to point to your adopted SD card and your previous internal storage data that actually lived in /data/media is moved over to its new home on the SD card. This gives the added benefit that there is NO ambiguity where apps are storing user data. It always goes to the adopted SD card. See appendix 4 for my hunch on why they encrypt the adopted card.

Xem thêm:   17 Great Search Engines You Can Use Instead of Google

Xem thêm :  phần mềm hát nhép

edit: There’s still a little confusion on adoptable storage. I’ll add the minute details to appendix 5. They are quite complex and don’t really change the overall concept. Suffice it to say, with adoptable storage, when you move an app to SD, all of its code and data is moved to SD, unlike pre-Marshmallow. If you read appendix 5, you’ll there is one exception, though.

Appendix 1 – The various folders inside /data You can not look at the raw files and folders inside this partition without rooting your device. I’ll list some of the interesting ones:

  • /data/app – This contains all user user-installed and updated system apps. Each app gets a folder by its package name (org.mozilla.firefox). Inside the folder, there is a base.apk which is what you download. A lib folder for any libraries. There is also, curiously, on devices with ART, an “oat” file. This is the compiled version of the app. The oat file is usually as big as the apk and lib combined. However, space consumed by oat files is not counted by my Moto G when looking at the storage & usb section of settings. This is absolutely ridiculous. My apps are effectively consuming twice the space that the phone tells me they are (before factoring space consumed by app data) and I had no idea where my space was going prior to rooting it.

  • /data/data – This contains the application’s private data that you can “clear” from the app management screen. It also gets automatically delete when you uninstall the app so it’s not a good candidate for storing data to persist across app installations (think chat apps like WhatsApp). This is why apps many times choose to litter the root directory of the user storage with the crap instead of storing it in the system managed directory. There is no way for an application to know when it’s about to be uninstalled and save some of its files, so they dump them all into user storage.

  • /data/dalvik-cache – This contains the dalvik VMs byte-code that is required for your app to run pre-Lollipop. Starting with Lollipop, it contains oat files that are binaries for ART. They just didn’t bother to rename the directory.

  • /data/tombstones – This contains the crash reports when your apps stop responding. I guess this folder will also grow over time, consuming increasing amounts of space until you reset your device. Since I just rooted my phone, I can’t be sure about this though since apps haven’t crashed yet. Somebody else may have better information about this.

  • /data/media – Ah. Here we are. This contains the files that you see when you use a file manager. You don’t even see your entire userdata partition! Just one folder in it. Actually it’s just one folder inside /data/media, typically /data/media/0. You see, Android is a multi-user operating system now. So, there have to be boundaries between how much data users can share and see. Any and all apps are shared system-wide. They are updated everywhere even if one user updates one. But, the user data is unique to every user on the device. This includes the app’s private data in /data/data and the “internal SD card” the user can see with file managers. So, the first user of the device will see /data/media/0, the second one will see /data/media/1 and so on… In fact, you don’t get to see /data/media/0 directly either (remember the user and apps have no access to /data)! It’s mounted to a different location (/storage/emulated) via the External SD Card Filesystem. This loop-back mount filesystem is what enforces application permissions such as WRITE_EXTERNAL_STORAGE from the SDK or the user-configurable “storage” permission in Marshmallow. There is one more folder in /data/media. It’s /data/media/obb. When you download huge games that download “additional data”, this is where it goes. It can not be user-specific, since it’s application code (or app resources like artwork but whatever). It’s curious to me why the obb folder lives in /data/media instead of just /data but I suspect that may be due to historical reasons since obb folder used to live on the first “SD Card” of the device (whether that was internal storage or an actual SD Card) before multi-user came along.

Appendix 2 – App data stored on SD card Your device’s “usb storage” has a folder called ‘Android’ on it. You can see it in any file manager even with an unmodified device. This folder is meant to hold app data that developers don’t mind exposing to the world. Applications don’t need any special permissions to write to their special folder inside ‘Android’. Note that while other applications CAN read data that one application puts in its ‘Android’ folder on the SD card, it’s not meant to be used to share user files like pictures and music. App data inside the ‘Android’ folder is still deleted when the application is uninstalled. Google used this folder as a way to provide reprieve to users that had limited device memory but lots of SD card space before the adoptable storage feature in Marshmallow. The idea was that if the application didn’t care that its data could be read by other apps and the user, it could put files on the SD card instead of straining the limited device storage. This was perfect for streaming music applications, for example, that supposedly encrypted their cached music files anyway. Now that adoptable storage is here, it is my opinion that it makes little sense to store app data this way and it should be phased out in future Android versions (causing yet another breaking API change with regards to storage…). This is because the SD card is either being used in adoptable storage mode, which means the user can move the entire application and its private data to the SD card anyway. Or, the SD card is being used as Portable Storage (more akin to how you’d use an SD card in your laptop or something). This would make it a poor location to put application data on since the user might remove the card at any time. Moreover, lots of apps just dump data to their special ‘Android’ folder on the first SD card, which is usually the device’s internal storage simulated as an SD card anyway so the benefit of saving space on the device is gone. See why it’s confusing? We’re better off without it in Android N.

Xem thêm:   How to do a Clean Install of Windows 10 the Easy Way

Xem thêm :  Soạn bài sang thu

Appendix 3 – How the native “move apps to SD” feature worked before Marshmallow Before adoptable storage in Marshmallow, Android allowed apps to be moved to the SD card in a limited capacity. Only if the developer marked their app as movable and only a FEW of the app’s files would be moved over. Specifically, the apps’ obb files always stayed on internal storage and they are some the most space consuming files! Moreover, apps frequently moved back to internal storage when they got updated from the Play store. This feature was more of hassle and I rarely bothered with it.

Appendix 4 – Why encrypt the SD card Before adoptable storage, when Android moved an app to SD the way appendix 3 describes, it would create a folder under .android_secure on your SD card. The app’s files under .android_secure would be INDIVIDUALLY encrypted to maintain app privacy because an SD card can always be browsed on a computer. Remember, Android guarantees app data stored in /data/data is private to the app on an unmodified device. I guess this isn’t very good for performance. With adoptable storage, they needed to store all of an app’s files and private data on the SD card. They could have done this with just formatting it as EXT4 but remember, an SD card can always be pulled out and put into a laptop. They didn’t want to expose apps’ private data this way probably, so they chose to encrypt the entire device. See this excellent blog post on recovering the encryption key for an adopted SD card: http://nelenkov.blogspot.in/2015/06/decrypting-android-m-adopted-storage.html You’ll need to be rooted, obviously.

Appendix 5 – More on adoptable storage The layout of the SD card once it’s adopted as internal storage looks a lot like the /data partition. The adopted SD card is actually mounted at /mnt/expand/$UUID. Some folders within this directory:

  • /app: Equivalent of /data/app from appendix 1 for the apps you move to SD card. This includes the apk, libs, and the compiled oat file. Individual apps and their data, can be kept on internal storage or moved to the SD card if allowed by the developer. You’re not forced to place them all in one place. If you do choose to move an app to SD, all of it and its associated data goes to the SD card.

  • /user/0: Equivalent of /data/data from appendix 1 for apps you move to SD. So this means an app’s private data is also moved to SD.

  • /media: Equivalent of /data/media from appendix 1. This is where it gets interesting. After /u/nandhp inquired in his comment, I looked into it more. I admit I completely forgot to look at this part the first time. You actually have the option to host the contents of /storage/emulated (the ‘user-accessible storage’ of your device as seen by file managers) on either /data/media (the device’s internal storage) or /mnt/expand/sd-card/media (the device’s adopted storage). It has to be one or the other – you can’t pool the storage for reasons mentioned above.

So, you see, an app is completely moved to SD, unlike previous implementations of the app to SD feature in Android. The only exception here is (it’s complicated…) the obb files. All apps’ obb files reside in /storage/emulated/obb. So, if your /storage is pointing to the internal partition, all apps’ obb files will reside on internal storage. If your /storage is pointing to the adopted storage, all apps’ obb files will be on the adopted storage, regardless of whether the app itself is on internal storage or the SD card. While this decision to house obb files under /storage is odd to me, it shouldn’t matter much because I think the most common use case will be to shove a huge SD card in the device and use it as your /storage/emulated.

 

Appendix 6 – adoptable storage benchmarks

 Internal storageAdoptable storage (class 10 card)Sequential read87.20 MiB/s32.31 MiB/sSequential write43.40 MiB/s18.74 MiB/sRandom readRandom write

 

So, is it worth it? Yes! I just threw a 64 GB card in there and adopted it. You have to pry open the water-resistant back cover to access the SD card on the Moto G, anyway, so I’d never remove it. If I want to transfer data, I just use a USB cable or something like Air Droid. Now, I don’t have to worry about if I’ll run out of space while shooting a 1080p video, for example. Games can usually be moved to SD also, along with their obb files so I effectively have infinite space for apps, pictures and music. It works well. Is it slow? Not really in Real World Usage – though I may be a little more forgiving on the .5 second delay in load times than the ones sporting an S7… The only downside, as I mentioned, is that developers still control whether their app is movable to SD. I don’t agree with this decision on an adopted storage device and hope it gets changed in the future.

Xem thêm bài viết thuộc chuyên mục: Tips – IT and Computing

Xem thêm bài viết thuộc chuyên mục: Kiến Thức Chung

Related Articles

Back to top button