Talk:Homebrew ABI: Difference between revisions

(10 intermediate revisions by 3 users not shown)

Line 37:

--[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 00:36, 6 November 2018 (UTC)

:"- The NRO file is not persisted to the SD card." "- The NRO file is not stored somewhere that the homebrew library knows how to access." "- The application does not exist as an NRO file at all." The ABI requires the NRO to located on SD in the first place... --[[User:Yellows8|Yellows8]] ([[User talk:Yellows8|talk]]) 19:34, 8 November 2018 (UTC)

:Like yellows8 said, the homebrew ABI is designed around NRO files stored on SD card. If we ever support other filesystems (such as hostio access), we'd just add support for host:/ paths being passed through argv (as well as support code in libnx which would be trivial) - homebrew apps would need a recompile anyway with or without new ABI keys. Also, for that use case (hostio), romfs doesn't really make sense because you could just use assets in a folder inside the host filesystem instead of having to go through the trouble of building romfs (which can get pretty large). Finally, homebrew by definition is NRO formatted. Other formats should not be supported or promoted. --[[User:Fincs|Fincs]] ([[User talk:Fincs|talk]]) 19:47, 8 November 2018 (UTC)

yellows8: It says nowhere in the homebrew ABI document that the NRO is required to be located on the SD card. If you are absolutely convinced that that should be an HBABI requirement, I would encourage you to add it to the document. I very much think that should not be an HBABI requirement though. The only things keeping homebrew on the SD card are libnx's behavior of reading its own executable combined with the small set of default-mounted filesystems that libnx understands. Aside from this behavior (which I'm trying to get changed), there is no reason to require that homebrew applications be located on the SD card.

fincs: Apologies if I was unclear. When I said RomFS, I meant the filesystem that is visible to libnx applications under <code>romfs:/*</code> paths. For the hostio use case, if hostio were to be mounted under <code>hostio:</code>, the application would need to be written to tell whether it's on hostio or not and adjust its asset loading paths accordingly, whereas it would be significantly more convenient if the homebrew loader could override the <code>romfs:</code> mountpoint to use a hostio IFileSystem. I also disagree that homebrew is "by definition" NRO formatted. By convention, sure, and I'd even accept that this ABI does not apply to non-NRO formatted homebrew. I invite you to ignore that use case.

Here's a different idea for you.

==== Mount ====

This is used to request that a filesystem be mounted by the homebrew application.

* '''Key:''' 15

* '''Value[0]:''' Pointer to a NULL-terminated string representing the desired mountpoint.

* '''Value[1]:''' Handle to a session implementing [[Filesystem_services#IFileSystem]].

The length of the mountpoint shall be no longer than 32 characters, including the NULL terminator.

This key may be combined with the Argv key to specify the location of an application's executable.

This way, we can still override <code>romfs:/</code> with Mount["romfs", <IFileSystem>], or we can mount an arbitrary filesystem (Mount["myfs", <IFileSystem>]) and set argv[0] to a path on that new mountpoint (<code>myfs:/application.nro</code>) so that the application can still find its NRO file and read it back to provide the <code>romfs:/</code> mount even if the NRO is not located on the SD card. I've even specified a length limit to match fsdev requirements.

--[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 23:42, 9 November 2018 (UTC)

[[Homebrew_ABI#NextLoadPath]] "NRO" "should start with "sdmc:/". Of course nx-hbloader doesn't enforce the latter besides only having sdmc mounted. --[[User:Yellows8|Yellows8]] ([[User talk:Yellows8|talk]]) 00:05, 10 November 2018 (UTC)

For NextLoadPath? Sure, if you want to use NextLoadPath, it's best to give it a path to an NRO starting with "sdmc:/" ''because that's all that nx-hbloader understands'' and all that you want it to understand, which makes sense. But HBABI doesn't require that homebrew needs to be loaded by passing NextLoadPath back to the loader. After all, nx-hbmenu isn't loaded that way. That's besides the point, anyway. The point is that whether or not this restriction is actually documented on the HBABI page, I want you to evaluate why this restriction exists and whether it's necessary or not.

--[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 00:27, 10 November 2018 (UTC)

Where does this "IFilesystem" come from? --[[User:Yellows8|Yellows8]] ([[User talk:Yellows8|talk]]) 01:47, 10 November 2018 (CET)

Doesn't much matter as far as the application is concerned. Could be some save data that the loader opened from FS, could be some BIS filesystem, or, in my case, a custom IFileSystem implementation that came from a custom service. Just as long as it implements the IFileSystem interface.

--[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 00:57, 10 November 2018 (UTC)

Workarounds (which only exist for a debugger) don't belong in the HBABI. --[[User:Yellows8|Yellows8]] ([[User talk:Yellows8|talk]]) 19:17, 12 February 2019 (UTC)

Almost all of HBABI is workarounds to facilitate running homebrew applications in dirty environments:

- bad entrypoint arguments (MainThreadHandle)

- reusing the same process (OverrideHeap, Argv (can't use loader-args), AllocPages, LockRegion, RandomSeed)

- legacy exploits (OverrideService, AppletWorkaround)

- working around 0xffff8001 check in SVCs (ProcessHandle)

And although it's not specified in the HBABI, reading back argv[0] for ASET is also a way to work around not being installed as a real application and being able to access romfs like official applications do.

The only thing I'm working around here is the ASET workaround that needlessly ties homebrew to the SD card. Even if this is a workaround for a debugger, I don't think that's grounds for not belonging in the HBABI. It's not like Nintendo doesn't do anything similar ("Redirect*" commands in lr).

--[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 21:42, 12 February 2019 (UTC)

Reusing the same process is the approach selected for nx-hbloader which is the standard way to load homebrew, and I see no reason why they would be labeled as "workarounds". AllocPages and LockRegion were unilaterally added by you several months ago and nx-hbloader has no use for them. OverrideService and AppletWorkaround are not presently used by anything and they were mostly added for future-proofing in case we need to support a more restrictive environment where we need to pass stolen service handles, they don't exist for "legacy" exploits.

I see no reason to add a romfs override key. If we want to support non-SD card paths, we will add other device name prefixes to argv[0] (such as host:/) and add support for these devices in libnx.

--[[User:Fincs|Fincs]] ([[User talk:Fincs|talk]]) 22:12, 12 February 2019 (UTC)

@@ Line 37: / Line 37: @@
 --[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 00:36, 6 November 2018 (UTC)
+:"- The NRO file is not persisted to the SD card." "- The NRO file is not stored somewhere that the homebrew library knows how to access." "- The application does not exist as an NRO file at all." The ABI requires the NRO to located on SD in the first place... --[[User:Yellows8|Yellows8]] ([[User talk:Yellows8|talk]]) 19:34, 8 November 2018 (UTC)
+:Like yellows8 said, the homebrew ABI is designed around NRO files stored on SD card. If we ever support other filesystems (such as hostio access), we'd just add support for host:/ paths being passed through argv (as well as support code in libnx which would be trivial) - homebrew apps would need a recompile anyway with or without new ABI keys. Also, for that use case (hostio), romfs doesn't really make sense because you could just use assets in a folder inside the host filesystem instead of having to go through the trouble of building romfs (which can get pretty large). Finally, homebrew by definition is NRO formatted. Other formats should not be supported or promoted. --[[User:Fincs|Fincs]] ([[User talk:Fincs|talk]]) 19:47, 8 November 2018 (UTC)
+yellows8: It says nowhere in the homebrew ABI document that the NRO is required to be located on the SD card. If you are absolutely convinced that that should be an HBABI requirement, I would encourage you to add it to the document. I very much think that should not be an HBABI requirement though. The only things keeping homebrew on the SD card are libnx's behavior of reading its own executable combined with the small set of default-mounted filesystems that libnx understands. Aside from this behavior (which I'm trying to get changed), there is no reason to require that homebrew applications be located on the SD card.
+fincs: Apologies if I was unclear. When I said RomFS, I meant the filesystem that is visible to libnx applications under <code>romfs:/*</code> paths. For the hostio use case, if hostio were to be mounted under <code>hostio:</code>, the application would need to be written to tell whether it's on hostio or not and adjust its asset loading paths accordingly, whereas it would be significantly more convenient if the homebrew loader could override the <code>romfs:</code> mountpoint to use a hostio IFileSystem. I also disagree that homebrew is "by definition" NRO formatted. By convention, sure, and I'd even accept that this ABI does not apply to non-NRO formatted homebrew. I invite you to ignore that use case.
+Here's a different idea for you.
+    ==== Mount ====
+    This is used to request that a filesystem be mounted by the homebrew application.
+    * '''Key:''' 15
+    * '''Value[0]:''' Pointer to a NULL-terminated string representing the desired mountpoint.
+    * '''Value[1]:''' Handle to a session implementing [[Filesystem_services#IFileSystem]].
+    The length of the mountpoint shall be no longer than 32 characters, including the NULL terminator.
+    This key may be combined with the Argv key to specify the location of an application's executable.
+This way, we can still override <code>romfs:/</code> with Mount["romfs", <IFileSystem>], or we can mount an arbitrary filesystem (Mount["myfs", <IFileSystem>]) and set argv[0] to a path on that new mountpoint (<code>myfs:/application.nro</code>) so that the application can still find its NRO file and read it back to provide the <code>romfs:/</code> mount even if the NRO is not located on the SD card. I've even specified a length limit to match fsdev requirements.
+--[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 23:42, 9 November 2018 (UTC)
+[[Homebrew_ABI#NextLoadPath]] "NRO" "should start with "sdmc:/". Of course nx-hbloader doesn't enforce the latter besides only having sdmc mounted. --[[User:Yellows8|Yellows8]] ([[User talk:Yellows8|talk]]) 00:05, 10 November 2018 (UTC)
+For NextLoadPath? Sure, if you want to use NextLoadPath, it's best to give it a path to an NRO starting with "sdmc:/" ''because that's all that nx-hbloader understands'' and all that you want it to understand, which makes sense. But HBABI doesn't require that homebrew needs to be loaded by passing NextLoadPath back to the loader. After all, nx-hbmenu isn't loaded that way. That's besides the point, anyway. The point is that whether or not this restriction is actually documented on the HBABI page, I want you to evaluate why this restriction exists and whether it's necessary or not.
+--[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 00:27, 10 November 2018 (UTC)
+Where does this "IFilesystem" come from? --[[User:Yellows8|Yellows8]] ([[User talk:Yellows8|talk]]) 01:47, 10 November 2018 (CET)
+Doesn't much matter as far as the application is concerned. Could be some save data that the loader opened from FS, could be some BIS filesystem, or, in my case, a custom IFileSystem implementation that came from a custom service. Just as long as it implements the IFileSystem interface.
+--[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 00:57, 10 November 2018 (UTC)
+Workarounds (which only exist for a debugger) don't belong in the HBABI. --[[User:Yellows8|Yellows8]] ([[User talk:Yellows8|talk]]) 19:17, 12 February 2019 (UTC)
+Almost all of HBABI is workarounds to facilitate running homebrew applications in dirty environments:
+  - bad entrypoint arguments (MainThreadHandle)
+  - reusing the same process (OverrideHeap, Argv (can't use loader-args), AllocPages, LockRegion, RandomSeed)
+  - legacy exploits (OverrideService, AppletWorkaround)
+  - working around 0xffff8001 check in SVCs (ProcessHandle)
+And although it's not specified in the HBABI, reading back argv[0] for ASET is also a way to work around not being installed as a real application and being able to access romfs like official applications do.
+The only thing I'm working around here is the ASET workaround that needlessly ties homebrew to the SD card. Even if this is a workaround for a debugger, I don't think that's grounds for not belonging in the HBABI. It's not like Nintendo doesn't do anything similar ("Redirect*" commands in lr).
+--[[User:Misson20000|Misson20000]] ([[User talk:Misson20000|talk]]) 21:42, 12 February 2019 (UTC)
+Reusing the same process is the approach selected for nx-hbloader which is the standard way to load homebrew, and I see no reason why they would be labeled as "workarounds". AllocPages and LockRegion were unilaterally added by you several months ago and nx-hbloader has no use for them. OverrideService and AppletWorkaround are not presently used by anything and they were mostly added for future-proofing in case we need to support a more restrictive environment where we need to pass stolen service handles, they don't exist for "legacy" exploits.
+I see no reason to add a romfs override key. If we want to support non-SD card paths, we will add other device name prefixes to argv[0] (such as host:/) and add support for these devices in libnx.
+--[[User:Fincs|Fincs]] ([[User talk:Fincs|talk]]) 22:12, 12 February 2019 (UTC)