Why?
Currently, DUI is only able to Accept mouse input into the browser, which heavily limits functionality.
Pretty much all big sites (google, YouTube, disk cord, etc) block being loaded into iframes, which completely removes the possibility of a script adding their own top-level JS to handle it.
Allowing Keyboard Input allows so many more possibilities to what you can accomplish with DUI rendering
Usage/Examples
Browsing YouTube
(using voice chats and playing games, would be cool to be able to send actual messages)
Watch [Mycroft] DUI Discord Showcase | Streamable
Other Cool use cases
https://streamable.com/n789ho
*Notable usage: browsing https://google.com/
As you can see in the amazing previews above, I’m able to use mouse input to do anything I want, but, when it comes down to needing text input, I cannot do anything, from logging into sites, sending messages, searching google and YouTube, Hell it can even render native 4k video and images but not even accept a simple key press?
Implementation Suggestion
User Experience
My Suggestion for Implementation is to use the same methods as SEND_DUI_MOUSE_DOWN
and SEND_DUI_MOUSE_UP
, and they would be used in the exact same way:
if IsDisabledControlJustPressed(0, 38) then --Pressed E
SendDuiKeyDown(duiObject, "e")
end
if IsDisabledControlJustReleased(0, 38) then --Released E
SendDuiKeyUp(duiObject, "e")
end
they can also be combined by the user like so:
function KeyPressed(dui, key)
SendDuiKeyDown(dui, key)
SendDuiKeyUp(dui, key)
end
KeyPressed(duiObject, "e")
Backend
As for the backend, my suggestion is:
.AddMethod("SEND_DUI_KEY_UP", &NUIWindowWrapper::InjectKeyUp)
.AddMethod("SEND_DUI_KEY_DOWN", &NUIWindowWrapper::InjectKeyDown)
Adding these to the pre-existing hooks/injections would work great,
switching out the existing,
browser->GetHost()->SendMouseMoveEvent(ev, false);
with
browser->GetHost()->SendKeyEvent(keyEvent);
Similar to the existing KeyEvent System for normal NUI inputs, and developers should easily be able to add keyboard input into their DUI without having to use hacky and dangerous methods.
Ending
Thanks for Reading, and I hope we can massively improve how amazing DUI is, and allow more amazing scripts to be made.