Support WITH (timeout/wait) for DDL statements

What type of enhancement is this?

Tech debt reduction, User experience

What does the enhancement do?

Enable all DDL procedures (CREATE/ALTER/DROP/TRUNCATE/COMMENT/VIEW/FLOW, and trigger where applicable) to accept WITH (timeout='...', wait=...) and propagate these options to the DDL executor, so callers can control async behavior consistently. Today only ALTER TABLE repartition supports these options; this change makes it uniform and predictable across all DDLs.

We can break this issue down into multiple subtasks:

• CREATE TABLE
• CREATE VIEW
• DROP TABLE
• DROP DATABASE
• CREATE FLOW
• TRUNCATE TABLE
• ...

Implementation challenges

• DDL options are currently mixed with table/database options via parse_with_options and OptionMap (see src/sql/src/parsers/utils.rs and src/table/src/requests.rs); for CREATE TABLE/CREATE DATABASE, timeout/wait must not be persisted as table/db options.
• parse_ddl_options exists but is only used in the repartition path (src/operator/src/statement/ddl.rs); the logic needs to be generalized and applied to all DDL submission paths.
• DdlOptions in src/common/meta/src/ddl_manager.rs is only used by repartition; other procedures lack a timeout mechanism and always wait for completion.
• Decide semantic behavior when wait=true and timeout triggers (return timeout error vs. procedure id), and ensure consistency with existing Repartition timeout semantics.

Research notes (current state)

• DDL option keys are defined in src/table/src/requests.rs (DDL_TIMEOUT, DDL_WAIT, VALID_DDL_OPTION_KEYS).
• SQL parser only attaches WITH options to ALTER TABLE (see src/sql/src/parsers/alter_parser.rs, includes TODO for DDL options); other DDL statements do not parse WITH.
• parse_ddl_options lives in src/operator/src/statement/ddl.rs and is only applied in the repartition flow; all other SubmitDdlTaskRequest creation paths ignore options.
• DdlManager uses DdlOptions only in repartition; other procedures always call execute_procedure_and_wait without timeout.
• Repartition procedure supports per-operation timeouts (src/meta-srv/src/procedure/repartition.rs), but other DDL procedures do not.

Proposed plan

1. SQL layer

   • Add WITH (...) DDL options to DDL statements that currently have none: DROP TABLE/DATABASE/FLOW/VIEW, TRUNCATE TABLE, COMMENT ON, CREATE/DROP VIEW/FLOW, CREATE/DROP DATABASE, etc.
   • Introduce a ddl_options: OptionMap field (or equivalent) in relevant AST structs, separate from table/db options.
   • Add parser helper to validate only wait/timeout for DDL options, and split them from table/db options where WITH already exists (e.g., CREATE TABLE/CREATE DATABASE).
   • Add parser tests for each statement (positive and invalid option cases).

2. Operator layer

   • Extend statement handling to extract DDL options and set SubmitDdlTaskRequest.wait/timeout consistently for every DDL submission path in src/operator/src/statement/ddl.rs.
   • Ensure DDL options are removed before building CreateTableExpr / TableOptions / database options.

3. Metasrv layer

   • Add a generic timeout path for wait=true requests (e.g., tokio::time::timeout around watcher::wait).
   • Propagate DdlOptions to procedures where needed; add procedure-level timeouts for long-running tasks (drop database, create/drop table, truncate table, flow/view ops) where external RPCs can block.

4. Tests and docs

   • Unit tests for parse_ddl_options and new statement parsing.
   • Integration tests for WAIT=false (returns procedure id) and TIMEOUT behavior.
   • Update docs/examples to show WITH (timeout, wait) on multiple DDLs.